A native macOS AI chat app powered by MLX. 100% local inference on Apple Silicon, no cloud required. Built with ShipSwift.
- app
- klee
- self-hosted
- apple-silicon
132,241 visible repositories in the index
97–120 of 233 · Most stars
A native macOS AI chat app powered by MLX. 100% local inference on Apple Silicon, no cloud required. Built with ShipSwift.
Router that virtually distributes inference across connected devices in the home.
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
Pure C inference of Mistral Voxtral Realtime 4B speech to text model
Route, manage, and analyze your LLM requests across multiple providers with a unified API interface.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
A high-performance inference engine for LLM, VLM and DiT models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
V100 / SM70-focused vLLM engineering fork for modern LLM inference.
Local inference engine optimized for NVIDIA Blackwell
Jev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.
Parallax is a distributed model serving framework that lets you build your own AI cluster anywhere
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.
Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀
OpenTelemetry Instrumentation for AI Observability
Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows
Jlama is a modern LLM inference engine for Java
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.