1.vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
+3,124 stars
from 92,954 · score 8.05
Trending repositories
Ranked by Attention Score: stars gained over the last 30 complete days
7 repositories by Attention Score · Oct 4, 2026 – Nov 2, 2026 (UTC)
2.bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
+1,437 stars
from 5,783 · score 7.27
3.NVIDIA / TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
+252 stars
from 14,735 · score 5.53
4.ray-project / ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
+251 stars
from 43,923 · score 5.53
5.skypilot-org / skypilot
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
+116 stars
from 10,661 · score 4.76
6.bentoml / OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
+35 stars
from 12,540 · score 3.58
7.superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
+8 stars
from 5,320 · score 2.20