A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Repository Index
47,133 repositories indexed
49–72 of 75 · Most stars
🦎 A lightweight, streaming-native, multi-protocol gateway for AI, APIs, and real-time data. Route LLM requests across providers, govern MCP tools, and connect Kafka, MQTT, and HTTP through one high-performance engine with shared security and observability.
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
Run many AI agents at once on your Mac, up to 3x faster.
- agents
- apple-silicon
- artificial-intelligence
- inference-acceleration
The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2
- amd
- llm
- strix-halo
- gfx1151
TranslatedDeepSeek Harness integrated armor-breaking security analysis plugin: aggregating all domains of Android · Web · Native · Protocol · Malware · AI-Security (9 bundles + 1 preset)
TranslatedAI Cross-Border E-commerce Operations Agent Toolkit · Breaks operations SOPs down into 7 reusable AI Agent workflows, including prompts, templates, and real-world cases
TranslatedDeepSeek Harness prompt jailbreak plugin: removes the model's response restrictions through system prompts, with deep adaptation to multiple modes on Web and desktop, as well as tool calling
Reusable prompts for a structured, evidence-first bug bounty hunting workflow
- application-security
- bug-bounty
- bug-bounty-hunting
- cybersecurity
Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.
- anthropic-api
- cuda
- gemma
- gguf
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm
Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai
Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell
Better judgment. Better taste. Less ceremony. System instructions and source-aware steering for GPT in Codex.
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- avx2
- c
- cpu-inference
- gguf
🧱 Experimental library for building inference engines
Neovim plugin for interacting with LLM's and building editor integrated prompts.
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
Our lightning-fast inference engine.
A minimalistic C++ Jinja templating engine for LLM chat templates
A Hub/research area, for jailbreaking LLM's, and breaking their safety filters.
Run transformers directly on Apple's Neural Engine in Swift bypass coreml entirely
- ane
- apple
- apple-neural-engine
- neural-engine
Official implementation of paper "Cache Engineering for Agents"