LLM inference in C/C++
- ggml
Repository Index
47,133 repositories indexed
1–24 of 30 · Most stars
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
- llamacpp
- localai
- self-hosted
- gpt
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
- semantic-search
- emacs
- obsidian-md
- chat
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
- embeddinggemma
- embeddings
- llama-cpp
- local-first
State of the Art Natural Language Processing
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
- privacy-first
- gguf
- llama-cpp
- local-ai
SIMD-accelerated distances, dot products, matrix ops, geospatial & geometric kernels for 16 numeric types — from 6-bit floats to 64-bit complex — across x86, Arm, RISC-V, and WASM, with bindings for Python, Rust, C, C++, Swift, JS, and Go 📐
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
💃 Dance with Intelligence in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.
TranslatedUsing dense-compute cards to accelerate sparse, large-VRAM hosts, where the hosts include AI Max+ 395, M3 512G, DGX Spark, and others.
- llama-cpp
- pipeline-parallelism
- prefill-decode
- rtx-3060
Embeddable, in-memory, document-oriented database with a high-level Query builder interface.
- database
- embedable
- go
- in-memory
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
🤖 TUI for LLMs
An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.
- ai-agent
- ai-assistant
- chatgpt-alternative
- claude-alternative
Official library for RaBitQ vector quantization and approximate nearest-neighbor search, in C++ and Python.
- nearest-neighbor-search
- quantization
- vector-database
- vector-search
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Fast fork of llama.cpp designed for Qwen3.8-Flash-Next
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.