TranslatedOpen-source manuscript of 《深入理解 AI Infra:量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
This is a project template which uses FastAPI, Pydantic 2.0, Alembic and async SQLModel as ORM. It shows a complete async CRUD using authentication and role base access control.
📚A curated list of Awesome Diffusion Inference Papers with Codes: Sampling, Cache, Quantization, Parallelism, etc.🎉
- deepcache
- diffusion
- dit
- sd15