Repository Index

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

• nlp
• embedding
• bert
• ernie

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

• kv-cache
• llm-inference
• local-llm
• quantization

Run GLM-5.3-Flash (321B), DeepSeek-V4.1-Flash (748B) and MiniMax-M3.1-Flash (428B) on your own GPUs - one engine, one dashboard, OpenAI- and Anthropic-compatible API.

• cuda
• glm
• glm-5
• llm

a systemone proxy using embeddings that distills a teacher model

[Tech Report] Simplifying and Improving On-Policy Distillation for Few-step Diffusion

• diffusion
• distillation
• few-step-generation