Repository Index

Distribute and run LLMs with a single file.

• cross-platform
• gguf
• llama-cpp
• local-ai

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

• kv-cache
• llm-inference
• local-llm
• quantization

Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.

• cuda
• glm
• glm-5
• llm

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

• godot
• godot-engine
• godot-plugin
• godot4

TranslatedMeetDo — open-source Meeting-to-Action AI by 筑听 (Zhuting, "build listen") · JKinco Listen. Self-hosted live meetings, local ASR/LLM, structured minutes, DOCX/PDF.

• docker
• fastapi
• funasr
• local-first

Modern desktop application (Rust + Tauri v2 + Svelte 5 + Candle (HF)) for communicating with AI models that runs completely locally on your computer. No subscriptions, no data sent to the internet — just you and your personal AI assistant

• ai-chat
• artificial-intelligence
• candle
• chatbot