Repository Index

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

• apple-silicon
• inference-server
• llm
• macos

A local inference engine for Apple silicon, built around the model.

• apple-silicon
• coding-agents
• llm-inference
• macos

Run many AI agents at once on your Mac, up to 3x faster.

• agents
• apple-silicon
• artificial-intelligence
• inference-acceleration

Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.

• apple-silicon
• llm
• llm-inference
• local-llm

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• apple-silicon
• cuda
• gguf
• inference

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

• ai
• gguf
• gpu
• inference

M-Courtyard: Local AI Model Fine-tuning Assistant for Apple Silicon. Zero-code, zero-cloud, privacy-first desktop app powered by Tauri + React + mlx-lm.

• ai-assistant
• apple-silicon
• desktop-app
• fine-tuning