Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Repository Index
76,356 visible repositories in the index
1–6 of 6 · Most stars
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
- knative
- machine-learning
- model-interpretability
- model-serving
Dockerized OpenAI-compatible wrapper for Kokoro-82M text-to-speech w/multiplatform CPU, AMD, NVIDIA GPU PyTorch; multi-speaker, clone-tuning, caption timestamps, SSML, websockets, optional readalong web UI
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
- ai
- api
- serving
- artificial-intelligence
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.
- machine-learning
- python
- gpu
- llms
Batched NVIDIA MotionBricks: generate the motion of hundreds of characters in one model call.
- character-animation
- crowd-simulation
- motion-generation
- motionbricks