LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Repository Index
47,133 repositories indexed
1–11 of 11 · Most stars
A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Run many AI agents at once on your Mac, up to 3x faster.
- agents
- apple-silicon
- artificial-intelligence
- inference-acceleration
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
M-Courtyard: Local AI Model Fine-tuning Assistant for Apple Silicon. Zero-code, zero-cloud, privacy-first desktop app powered by Tauri + React + mlx-lm.
- ai-assistant
- apple-silicon
- desktop-app
- fine-tuning
Run transformers directly on Apple's Neural Engine in Swift bypass coreml entirely
- ane
- apple
- apple-neural-engine
- neural-engine
A Long-Horizon Benchmark for Self-Improving Inference Systems
Capacitor plugin to run LLM models locally in IOS and Android, support AppleInteligence