1.vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
+593 stars
from 95,485 · score 6.39
Trending repositories
Ranked by Attention Score: stars gained over the last 7 complete days
8 repositories by Attention Score · Oct 27, 2026 – Nov 2, 2026 (UTC)
2.bojieli / ai-infra-book
《深入理解 AI Infra：量化分析与系统设计》（李博杰 著）开源书稿：从硬件约束和模型架构出发，量化推导 LLM 推理与训练系统设计。含全书正文、PDF、配套计算工具与实验
+250 stars
from 6,970 · score 5.53
3.ekzhang / resin
🧱 Experimental library for building inference engines
+132 stars
from 230 · score 4.89
4.ray-project / ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
+52 stars
from 44,122 · score 3.97
5.NVIDIA / TensorRT-LLM
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
+42 stars
from 14,945 · score 3.76
6.skypilot-org / skypilot
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
+29 stars
from 10,748 · score 3.40
7.bentoml / OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
+8 stars
from 12,567 · score 2.20
8.superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
+1 stars
from 5,327 · score 0.69