1.bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
+250 stars
from 6,970 · score 5.53
Trending repositories
Ranked by Attention Score: stars gained over the last 7 complete days
9 repositories by Attention Score · Oct 27, 2026 – Nov 2, 2026 (UTC)
2.syv-ai / HyperQwen
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
+243 stars
from 2,231 · score 5.50
3.StartLuxLabs / StartLux-Prism
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
+154 stars
from 74 · score 5.04
4.eagle-hq / EAGLE-4
Official Inference Code of EAGLE-4
+114 stars
from 989 · score 4.74
5.datawhalechina / zero-to-sglang
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
+59 stars
from 1,616 · score 4.09
6.ray-project / ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
+52 stars
from 44,122 · score 3.97
7.Lightning-AI / litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
+13 stars
from 13,728 · score 2.64
8.bentoml / OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
+8 stars
from 12,567 · score 2.20
9.superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
+1 stars
from 5,327 · score 0.69