A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
- ai
- api
- serving
- artificial-intelligence
132,241 visible repositories in the index
169–192 of 748 · Most stars
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
State-of-the-art LLM compression, built for production inference with vLLM
TurboDiffusion: 100–200× Acceleration for Video Diffusion Models
Run GPT model on the browser with WebGPU. An implementation of GPT inference in less than ~1500 lines of vanilla Javascript.
A collection of scientific methods, processes, algorithms, and systems to build stories & models.
The repository provides code for running inference with the Meta Segment Anything Audio Model (SAM-Audio), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
1 API key. 220,000+ AI models. Bytez is the largest serverless Model Inference API on the internet. Let's build ✨
The repository provides code for running inference with the SAM 3D Body Model (3DB), links for downloading the trained model checkpoints and datasets, and example notebooks that show how to use the model.
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
SoulX-Podcast is an inference codebase by the Soul AI team for generating high-fidelity podcasts from text.
Your Cheat Sheet for AI Engineering Interview – Questions and Answers.
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.
Official codebase for "Self Forcing: Bridging Training and Inference in Autoregressive Video Diffusion" (NeurIPS 2025 Spotlight)
Hummingbird compiles trained ML models into tensor computation for faster inference.
Open-source inference server and production cluster for all the models your agent needs.
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
校招、秋招、春招、实习好项目!带你从零实现一个高性能的深度学习推理库,支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step
Causal Inference for the Brave and True. A light-hearted yet rigorous approach to learning about impact estimation and causality.
ReactJS Maps without external dependencies
Vietnamese TTS with instant voice cloning • On-device • Real-time CPU inference • 48kHz audio quality • Chuyển văn bản thành giọng nói tiếng Việt • Text to speech tiếng Việt • TTS tiếng Việt
Building a modern functional compiler from first principles.
A static, type inferred and embeddable language written in Rust.
Make videos on the computer you already own. FreeVideo runs MiniMax H3 in as little as 8 GB of VRAM and 16 GB of RAM, and adapts its acceleration path to your hardware.
High-performance single-GPU inference for selected model checkpoints and GPUs.