Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Repository Index
132,241 visible repositories in the index
73–96 of 748 · Most stars
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
tsParticles - Easily create highly customizable JavaScript particles effects, confetti explosions and fireworks animations and use them as animated backgrounds for your website. Ready to use components available for React.js, Vue.js (2.x and 3.x), Angular, Svelte, jQuery, Preact, Inferno, Solid, Riot and Web Components.
- particles
- typescript
- npm
- javascript
Deep universal probabilistic programming with Python and PyTorch
- python
- pytorch
- machine-learning
- bayesian
The modular and type safe schema library for validating structural data 🤖
- bundle-size
- modular
- parsing
- schema
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
- model-serving
- mlops
- llmops
- generative-ai
AlphaFold 3 inference pipeline.
A Datacenter Scale Distributed Inference Serving Framework
- diffusion
- disaggregated-serving
- kubernetes
- llm-inference
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
- chatgpt
- deep-learning
- instruction-following
- language-model
DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
- causal-inference
- machine-learning
- graphical-models
- bayesian-networks
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
- cuda-kernels
- deepspeed
- fastertransformer
- llm-inference
TranslatedOpen-source manuscript of 《深入理解 AI Infra:量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Diffusion model(SD,Flux,Wan,Qwen Image,Z-Image,Minimax H3,...) inference in pure C/C++
- ai
- cplusplus
- diffusion
- ggml
TranslatedA collection of the Android SDK needed for Android development, tools used in development, Android development tutorials, Android design guidelines, free design resources, and more.
- android-sdk
- android-studio
- sdk-manager
- android-development
Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.
A framework for efficient model inference with omni-modality models
- diffusion
- inference
- model-serving
- pytorch
Fast, flexible LLM inference
- llm
- rust
- uqff
Open-source implementation of AlphaEvolve
- alphacode
- coding-agent
- deepmind
- deepmind-lab
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
- machine-learning
- scale
- workflow
- mlops
Infrastructure for continually self‑improving agents
Translated💎1MB lightweight face detection model
Home of StarCoder: fine-tuning & inference!
Gemma 4 26B-A4B inference in ~2 GB of RAM on any M-series MacBook
- gemma
- gpgpu
- llm
- local-ai