A Datacenter Scale Distributed Inference Serving Framework
- diffusion
- disaggregated-serving
- kubernetes
- llm-inference
140,194 visible repositories in the index
1–3 of 3 · Most stars
A Datacenter Scale Distributed Inference Serving Framework
Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2
聚合ChatGPT官方版、ChatGPT免费版、文心一言、Poe、chatchat等多平台,支持自定义导入平台