A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
54,641 visible repositories in the index
1–2 of 2 · Most stars
A framework for efficient model inference with omni-modality models
- diffusion
- inference
- model-serving
- pytorch