A Datacenter Scale Distributed Inference Serving Framework
- diffusion
- disaggregated-serving
- kubernetes
- llm-inference
96,838 visible repositories in the index
1–3 of 3 · Most stars
A Datacenter Scale Distributed Inference Serving Framework
Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.
Distributed inference for mobile, desktop and server.