Ollama

Description

Ollama lets you download, run, and manage large language models entirely on your own hardware. It packages model weights, configuration, and a runtime into a single command, giving developers a private, offline alternative to cloud LLM APIs for chat, coding assistance, and embeddings.

Features

• One-Command Model Pulls: Downloads and runs models like Llama, Qwen, and DeepSeek with a single command.
• Local Inference: Runs entirely on-device, so prompts and data never leave your machine.
• REST API: Exposes an OpenAI-compatible API for easy integration into existing apps.
• GPU Acceleration: Automatically uses available CUDA, Metal, or ROCm hardware for speed.
• Model Library: Provides a registry of pre-quantized, ready-to-run open models.
• Custom Modelfiles: Lets you fine-tune prompts, parameters, and system behavior per model.

Technology Stack

Core runtime written in Go, built on top of the llama.cpp inference engine. Ships official clients and integrations for Python, JavaScript, LangChain, and dozens of community desktop and chat UIs.

Requirements

macOS, Linux, or Windows; at least 8 GB RAM for small models (larger models need 16-64 GB or a dedicated GPU); NVIDIA, AMD, or Apple Silicon GPU recommended for acceleration; several GB of free disk space per model.