1.vllm-project / vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
+593 stars
from 95,485 · score 6.39
Trending repositories
Ranked by Attention Score: stars gained over the last 7 complete days
35 repositories by Attention Score · Oct 27, 2026 – Nov 2, 2026 (UTC)
2.vladimirrott / mojo-dllm
Local inference for diffusion language models (LLaDA, Dream), written in Mojo
+307 stars
from 3 · score 5.73
3.FlashML-org / FreeVideo
Make videos on the computer you already own. FreeVideo runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, and adapts its acceleration path to your hardware.
+241 stars
from 2,332 · score 5.49
4.SYSTRAN / faster-whisper
Faster Whisper transcription with CTranslate2
+240 stars
from 26,134 · score 5.48
5.sgl-project / sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
+192 stars
from 38,156 · score 5.26
6.ggml-org / whisper.cpp
Port of OpenAI's Whisper model in C/C++
+179 stars
from 54,667 · score 5.19
7.fudan-generative-vision / Remory
Remory: Learning Residual Memory for Context Compaction
+151 stars
from 490 · score 5.02
8.ekzhang / resin
🧱 Experimental library for building inference engines
+132 stars
from 230 · score 4.89
9.GeeeekExplorer / nano-vllm
Nano vLLM
+128 stars
from 16,125 · score 4.86
10.google-ai-edge / mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
+104 stars
from 37,401 · score 4.65
11.LMCache / LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
+92 stars
from 12,164 · score 4.53
12.T-Crypt / berth
Berth is a lightweight GPU resource manager written in Go. It controls which applications get access to your GPU, when they get it, and how that access is handed off.
+84 stars
from 182 · score 4.44
13.StartLuxLabs / StartLux-Decision
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
+74 stars
from 1,558 · score 4.32
14.Human-Agent-Society / reef
Infrastructure for continually self‑improving agents
+59 stars
from 7,727 · score 4.09
15.stas00 / ml-engineering
Machine Learning Engineering Open Book
+55 stars
from 19,589 · score 4.03
16.TrenTorch / TrenTorch
Don't memorize ML. Understand it from first principles including Inference, Cuda, Deep Learning, Data Science and Reinforcement Learning
+51 stars
from 713 · score 3.95
17.theagentrouter / agent-router
Manages Unified Access to Generative AI Services built on Envoy Gateway
+38 stars
from 2,290 · score 3.66
18.openvinotoolkit / openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
+36 stars
from 11,066 · score 3.61
19.ultralytics / yolov5
Ultralytics YOLOv5 in PyTorch for object detection, instance segmentation, classification, training, and export.
+26 stars
from 58,221 · score 3.30
20.Tencent / ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
+20 stars
from 23,997 · score 3.04
21.heterodoxin / AbliteratedOutTheBox
llama.cpp fork that abliterates at inference
+20 stars
from 142 · score 3.04
22.NVIDIA / TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
+11 stars
from 13,416 · score 2.48
23.nicorobinreal / llmspeed
Feel how fast a local LLM runs before you buy the hardware
+11 stars
from 156 · score 2.48
24.triton-inference-server / server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
+10 stars
from 11,112 · score 2.40
25.deepspeedai / DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
+8 stars
from 43,273 · score 2.20
26.gvergnaud / ts-pattern
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
+8 stars
from 15,198 · score 2.20
27.ultralytics / yolov3
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
+7 stars
from 10,624 · score 2.08
28.uxlfoundation / oneDNN
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
+4 stars
from 4,080 · score 1.61
29.defilantech / LLMKube
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
+4 stars
from 245 · score 1.61
30.hpcaitech / ColossalAI
Making large AI models cheaper, faster and more accessible
+3 stars
from 41,430 · score 1.39
31.aws / amazon-sagemaker-examples
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
+3 stars
from 10,998 · score 1.39
32.modelsphere / modelsphere
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.
+3 stars
from 168 · score 1.39
33.RunanywhereAI / runanywhere-sdks
Production ready toolkit to run AI locally
+1 stars
from 10,331 · score 0.69
34.superduper-io / superduper
Superduper: End-to-end framework for building custom AI applications and agents.
+1 stars
from 5,327 · score 0.69
35.typedb / typeql
TypeQL: Built for systems, not records
+1 stars
from 256 · score 0.69