Repository Index

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

• apple-silicon
• inference-server
• llm
• macos

High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model

A local inference engine for Apple silicon, built around the model.

• apple-silicon
• coding-agents
• llm-inference
• macos

Run many AI agents at once on your Mac, up to 3x faster.

• agents
• apple-silicon
• artificial-intelligence
• inference-acceleration

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

• apple-silicon
• llm
• llm-inference
• local-llm