A fast inference library for running LLMs locally on modern consumer-class GPUs
Repository Index
76,855 visible repositories in the index
1–2 of 2 · Most stars
Control panel for VLLM, Sglang, llama.cpp, exllamav3
76,855 visible repositories in the index
1–2 of 2 · Most stars
A fast inference library for running LLMs locally on modern consumer-class GPUs
Control panel for VLLM, Sglang, llama.cpp, exllamav3