20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Repository Index
47,133 repositories indexed
25–48 of 71 · Most stars
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
The repository provides code for running inference and finetuning with the Meta Segment Anything Model 3 (SAM 3), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Spark-TTS Inference Code
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
Bayesian Modeling and Probabilistic Programming in Python
- python
- statistical-analysis
- bayesian-inference
- mcmc
Deep universal probabilistic programming with Python and PyTorch
- python
- pytorch
- machine-learning
- bayesian
AlphaFold 3 inference pipeline.
Infrastructure for continually self‑improving agents
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Awesome autocompletion, static analysis and refactoring library for python
- static-analysis
- auto-complete
- python
- refactoring
Uplift modeling and causal inference with machine learning algorithms
- incubation
- machine-learning
- causal-inference
- uplift-modeling
Superduper: End-to-end framework for building custom AI applications and agents.
- ai
- mlops
- torch
- transformers
Python Toolkit for Causal and Probabilistic Reasoning
- python
- bayesian-networks
- causal-inference
- causal-discovery
Make videos on the computer you already own. FreeVideo runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, and adapts its acceleration path to your hardware.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
Official PyTorch inference for Breeze TTS 2
Pika H3 Flex: video and audio generation inference
Deep Object Pose Estimation (DOPE) – ROS inference (CoRL 2018)