A fast inference library for running LLMs locally on modern consumer-class GPUs
Repository Index
76,855 visible repositories in the index
1–4 of 4 · Most stars
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
Control panel for VLLM, Sglang, llama.cpp, exllamav3
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs