47,133 repositories indexed
1–6 of 6 · Most stars
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
TranslatedA Golang implemented Redis Server and Cluster.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.