Repository Index

A high-throughput and memory-efficient inference and serving engine for LLMs

• gpt
• llm
• pytorch
• model-serving

SGLang is a high-performance serving framework for large language models and multimodal models.

• cuda
• inference
• llama
• llm

[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device.

• ai
• faiss
• langchain
• llama-index

🧱 Experimental library for building inference engines

• inference
• llm
• cuda
• gpt-oss