A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
54,641 visible repositories in the index
1–2 of 2 · Most stars
An open, programmable decision layer for models and compute.
- mixture-of-models
- semantic-router
- vllm
- ai-gateway