The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Repository Index
97,836 visible repositories in the index
1–2 of 2 · Most stars
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.