LLM inference in C/C++
- ggml
Repository Index
47,133 repositories indexed
1–24 of 38 · Most stars
Port of OpenAI's Whisper model in C/C++
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
- llamacpp
- localai
- self-hosted
- gpt
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
- semantic-search
- emacs
- obsidian-md
- chat
Privacy-first AI meeting note taker for Windows and macOS. Free, open source (MIT). On-device Whisper/Parakeet transcription and speaker diarization, no meeting bot. Open-source alternative to Granola, Otter and Fireflies. https://meetily.ai
- meeting-minutes
- meeting-notes
- llm
- mac
A modern replacement for Redis and Memcached
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Production ready toolkit to run AI locally
~95% on SimpleQA (e.g. Qwen3.6-27B on a 3090). Supports all local and cloud LLMs (llama.cpp, Ollama, Google, ...). 10+ search engines - arXiv, PubMed, your private documents. Everything Local & Encrypted.
- academia
- arxiv
- brave
- deep-research
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
- embeddinggemma
- embeddings
- llama-cpp
- local-first
The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.
- ai-native
- nearest-neighbor-search
- rag
- vector-search
State of the Art Natural Language Processing
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
- privacy-first
- gguf
- llama-cpp
- local-ai
SIMD-accelerated distances, dot products, matrix ops, geospatial & geometric kernels for 16 numeric types — from 6-bit floats to 64-bit complex — across x86, Arm, RISC-V, and WASM, with bindings for Python, Rust, C, C++, Swift, JS, and Go 📐
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
💃 Dance with Intelligence in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.
使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。
- llama-cpp
- pipeline-parallelism
- prefill-decode
- rtx-3060
Embeddable, in-memory, document-oriented database with a high-level Query builder interface.
- database
- embedable
- go
- in-memory
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts