A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–18 of 18 · Most stars
Ultralytics YOLOv5 in PyTorch for object detection, instance segmentation, classification, training, and export.
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Making large AI models cheaper, faster and more accessible
SGLang is a high-performance serving framework for large language models and multimodal models.
Faster Whisper transcription with CTranslate2
Machine Learning Engineering Open Book
Nano vLLM
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
Infrastructure for continually self‑improving agents
Superduper: End-to-end framework for building custom AI applications and agents.
Make videos on the computer you already own. FreeVideo runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, and adapts its acceleration path to your hardware.
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
Don't memorize ML. Understand it from first principles including Inference, Cuda, Deep Learning, Data Science and Reinforcement Learning
Remory: Learning Residual Memory for Context Compaction
🧱 Experimental library for building inference engines