🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
Repository Index
90,527 visible repositories in the index
1–3 of 3 · Most stars
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia CUDA, AMD ROCm, Intel XPU, Tenstorrent, Huawei Ascent NPU, and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
- gptq
- optimum
- peft
- quantization
A handy CLI in Docker container to make optimum use of git tips, see: https://github.com/git-tips/tips