LLM inference in C/C++
- ggml
Repository Index
47,133 repositories indexed
1–24 of 24 · Most stars
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
- llamacpp
- localai
- self-hosted
- gpt
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
- semantic-search
- emacs
- obsidian-md
- chat
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
Unified framework for building enterprise RAG pipelines with small, specialized models
- parsing
- retrieval-augmented-generation
- agents
- generative-ai-tools
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.
- embeddinggemma
- embeddings
- llama-cpp
- local-first
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
The Swiss Army Knife of Offline AI. Chat, see, speak, and generate images on your phone or Mac — GGUF LLMs, vision, Whisper speech-to-text, Stable Diffusion, tool calling, and local-network servers. Runs on your CPU, GPU, or NPU. No account, no API key, zero data leaves your device.
- privacy-first
- gguf
- llama-cpp
- local-ai
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
💃 Dance with Intelligence in Your Code. Minuet offers code completion as-you-type from popular LLMs including OpenAI, Gemini, Claude, Ollama, Llama.cpp, Codestral, and more.
TranslatedUsing dense-compute cards to accelerate sparse, large-VRAM hosts, where the hosts include AI Max+ 395, M3 512G, DGX Spark, and others.
- llama-cpp
- pipeline-parallelism
- prefill-decode
- rtx-3060
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Rewrites AI drafts so they read like a person wrote them — runs locally, keeps the facts
🤖 TUI for LLMs
An easier way to use AI on your own computer. A free, open-source alternative to Claude and ChatGPT that runs open-weight models like Gemma and gpt-oss locally on Mac, Windows and Linux. Private, and works offline.
- ai-agent
- ai-assistant
- chatgpt-alternative
- claude-alternative
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Fast fork of llama.cpp designed for Qwen3.8-Flash-Next
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
TranslatedBig Blue Fish desktop pet with local LLM. A Windows AI desktop pet with a built-in 4B local large model. It uses a whale-girl persona that combines weather, tech news and the current application to keep you gentle company through text bubbles. Supports one-click installation and pausing inference when off power.
- ai-companion
- deepseek
- desktop-companion
- desktop-pet
llama.cpp fork that abliterates at inference
ReAlign models instantly, on demand. Per-request alignment for verified users and channels, or Always On. Use directly or with Hermes, OpenClaw and custom harnesses.
🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift
localaik is a local compatibility server for a subset of the Gemini and OpenAI APIs. Run one container locally, point your SDK to it, and the proxy serves both protocol shapes on the same port for tests and development.