LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Repository Index
132,241 visible repositories in the index
1–21 of 21 · Most stars
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
- gemma
- gpgpu
- llm
- local-ai
Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.
- apple-silicon
- local-llm
- macos
- mlx
MiniMax H3 inference engine for Mac computers
The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.
- apple-silicon
- local-ai
- mlx
- mtp
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
- agent
- anthropic-api
- apple-silicon
- claude-code
A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Executable file for VITS inference
A native macOS AI chat app powered by MLX. 100% local inference on Apple Silicon, no cloud required. Built with ShipSwift.
- app
- klee
- self-hosted
- apple-silicon
MixTeX multimodal LaTeX, ZhEn, and, Table OCR. It performs efficient CPU-based inference in a local offline on Windows.
MLX Studio - Easiest way to run LLM's on your Mac. All in one engine.
- llm
- lmstudio
- macbook
- mlx
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
- apple-silicon
- clover
- coding-agents
- decision-models
Video Frame Interpolation & Super Resolution using NVIDIA's TensorRT & Tencent's NCNN inference, beautifully crafted and packaged into a single app
- artificial-intelligence
- cain
- dpir
- frame-interpolation
Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.
- apple-silicon
- claude-code
- codex
- dflash2
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm
Dyno Lab — an AI safety, alignment and interpretability research workbench for Apple Silicon. Explore activations, probes, SAEs and interventions with local inference, a Python SDK, APIs and MCP.
- ai
- ai-interpretability
- ai-safety
- aisafety
Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.
- cuda
- desktop-app
- gguf
- llm
PROMETHEVS: run the full DeepSeek-V4.1-Flash-1017 (510 GB) on a gaming PC with 64 GB RAM and a 12 GB GPU. PROMETHEVS engine (compress-then-correct), OpenAI-compatible local server, report and logs.
Solana DeFi Security: On-Chain Governance Risk Analysis for Multisig, Timelock, and Upgrade Authorit
- angulardart
- ark
- btc
- cam
Inferred Model-based Fuzzer
- fuzzer
- kernel-fuzzing
- macos