Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Repository Index
65,764 visible repositories in the index
1–2 of 2 · Most stars
A framework for efficient model inference with omni-modality models
- diffusion
- inference
- model-serving
- pytorch