A fast inference library for running LLMs locally on modern consumer-class GPUs
Repository Index
76,356 visible repositories in the index
1–3 of 3 · Most stars
A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.
Control panel for VLLM, Sglang, llama.cpp, exllamav3