Description

Jan is an on-device GPT alternative that runs large language models locally for full user control. Built with privacy as a core principle, it operates without external servers. It supports multi-hardware setups including NVIDIA GPUs and Apple M-series. Developed using Cortex.cpp for optimal performance and efficiency. Jan also provides custom extensions and an OpenAI-like API interface.

Features

• Local AI & Privacy: Host powerful LLMs (Llama, Gemma, Mistral, Qwen) directly on your machine, ensuring sensitive data stays offline.
• GPU Acceleration: Harness NVIDIA CUDA for faster model inference and Apple M-series performance.
• Remote AI Integrations: Connect to services like Groq and OpenRouter while preserving data ownership.
• OpenAI-Compatible API: Seamlessly switch from OpenAI’s endpoints to Jan’s local server.
• Custom Extensions: Tailor your AI with specialized plugins and integrations.

Technology Stack

• C++ Engine (Cortex.cpp): Cross-platform local inference engine for on-device LLMs
• Node, Yarn, Make: Used for building and deploying Jan’s desktop and CLI applications
• GPU & Multi-Arch Support: NVIDIA, Apple M-series, Windows, MacOS, and Linux
• OpenAI-Compatible Endpoints: Easily integrate existing AI apps or services

Requirements

• MacOS: 13 or higher
• Windows: 10 or higher
  • Optional GPU support for NVIDIA (CUDA 11.7+ and driver 470.63.01+)
• Linux:
  • glibc 2.27+ and gcc/g++ 11+
  • Optional GPU support for NVIDIA (CUDA 11.7+)