LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Repository Index
132,241 visible repositories in the index
1–24 of 28 · Most stars
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
- gemma
- gpgpu
- llm
- local-ai
The free AI already on your Mac. CLI tool, OpenAI-compatible server, and interactive chat — all on-device via Apple Intelligence. No API keys, no cloud, no downloads.
- apple-intelligence
- apple-silicon
- cli
- foundationmodels
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
- llava
- llm
- mlx
- vision-transformer
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
- ai
- amd-gpu
- apple-silicon
- asr
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
- apple-silicon
- local-llm
- macos
- mlx
Hit your Claude Code usage limit? Keep going free and offline on a local model: Qwen 3.8, Gemma 4, and Qwen 4 the day it ships. Runs on Apple Silicon Macs (MLX), Linux and Windows (llama.cpp) with no API key. Your code stays home.
- ai-privacy
- anthropic
- apple-silicon
- claude-code
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.
- apple-silicon
- local-ai
- mlx
- mtp
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
- agent
- anthropic-api
- apple-silicon
- claude-code
A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer. Join our Discord: https://discord.com/invite/8wGSsvmg4V
Community maintained hardware plugin for vLLM on Apple Silicon
A native macOS AI chat app powered by MLX. 100% local inference on Apple Silicon, no cloud required. Built with ShipSwift.
- app
- klee
- self-hosted
- apple-silicon
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Nativ is an open-source local AI app for Mac. It runs open AI models on Apple Silicon, and nothing leaves your Mac.
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.
- apple-silicon
- deep-learning
- huggingface
- large-language-models
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.
- llm
- lmstudio
- macbook
- mlx
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
- apple-silicon
- clover
- coding-agents
- decision-models
⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
MimikaStudio - A local-first application for macOS (Apple Silicon) + Agentic MCP Support
Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.
- apple-silicon
- claude-code
- codex
- dflash2
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm