LLM inference in C/C++
- ggml
Repository Index
47,133 repositories indexed
1–24 of 31 · Most stars
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
- llamacpp
- localai
- self-hosted
- gpt
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
- semantic-search
- emacs
- obsidian-md
- chat
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Production ready toolkit to run AI locally
~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
- academia
- arxiv
- brave
- deep-research
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
- embeddinggemma
- embeddings
- llama-cpp
- local-first
The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.
- ai-native
- nearest-neighbor-search
- rag
- vector-search
State of the Art Natural Language Processing
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
- privacy-first
- gguf
- llama-cpp
- local-ai
SIMD-accelerated distances, dot products, matrix ops, geospatial & geometric kernels for 16 numeric types — from 6-bit floats to 64-bit complex — across x86, Arm, RISC-V, and WASM, with bindings for Python, Rust, C, C++, Swift, JS, and Go 📐
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
💃 Dance with Intelligence in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.
使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
- llama-cpp
- pipeline-parallelism
- prefill-decode
- rtx-3060
Embeddable, in-memory, document-oriented database with a high-level Query builder interface.
- database
- embedable
- go
- in-memory
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
🤖 TUI for LLMs
An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.
- ai-agent
- ai-assistant
- chatgpt-alternative
- claude-alternative
Official library for RaBitQ vector quantization and approximate nearest-neighbor search, in C++ and Python.
- nearest-neighbor-search
- quantization
- vector-database
- vector-search