Inference code for Llama models
Repository Index
47,133 repositories indexed
1–12 of 12 · Most stars
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
Translated🎭 277 plug-and-play AI expert roles — supporting 20 tools such as Claude Code/Cursor/Copilot, covering 20 departments including engineering/design/marketing/finance. Includes 64 original agents for the Chinese market (Xiaohongshu/Douyin/WeChat/Feishu/DingTalk/Qt host computer software/mechanical design). Paired with the orchestrator agency-orchestrator, a single sentence lets multiple experts collaborate automatically as a DAG.
TranslatedLangGPT: Empowering everyone to become a prompt expert! 🚀 📌 Proposer of Structured Prompt 📌 Initiator of Meta-Prompt 📌 The most popular paradigm for putting prompts into practice | Language of GPT The pioneering framework for structured & meta-prompt design 10,000+ ⭐ | Battle-tested by thousands of users worldwide Created by 云中江树 (Yunzhong Jiangshu, "river and trees amid the clouds")
The fast, precise document-intelligence engine for every language: text, tables and metadata from 100+ formats, with OCR, transcription and code intelligence.
- text-extraction
- document-intelligence
- metadata-extraction
- pdf-extraction
A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Run a 105 GB AI model on a Mac that can't hold it. Qwen3.8-Flash-Next 4-bit, streamed from SSD on 16–64 GB Apple Silicon Macs. Fully native Swift, MLX and Metal. Offline, no Python. CLI, Swift library, and Ollama/OpenAI/Anthropic APIs. Works with Claude Code and Codex.
- apple-silicon
- llm
- llm-inference
- local-llm
Scikit-learn for Recursive Self-improvement (RSI). A modular framework for recursive harness evolution.
- meta-learning
- reinforcement-learning
- rsi
- ai-agents
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift
Trust metadata for AI agents — a stamp costs ~15 tokens, re-verifying costs 15,000. Agents stamp what they verify; the next agent runs 'akf check' and builds on it. pip install akf