Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell
Repository Index
47,133 repositories indexed
25–31 of 31 · Most stars
Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.
DeepSeek-V4.1-Flash-1017 EXL3 on 2x DGX Sparks with TensorFold
🧱 Experimental library for building inference engines
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
Official implementation of paper "Cache Engineering for Agents"
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.