Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Repository Index
63,766 visible repositories in the index
1–3 of 3 · Most stars
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
- knative
- machine-learning
- model-interpretability
- model-serving
A minimal Python framework for building custom AI inference servers with full control over logic, batching, and scaling.
- ai
- api
- serving
- artificial-intelligence