2bit quants, kernels and runtime for consumer devices.
Repository Index
111,085 visible repositories in the index
1–4 of 4 · Most stars
llama.cpp fork with additional SOTA quants and improved performance
KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM
- ggml
- kv-cache
- llama-cpp
- llm-inference
C++ backend for the jupyter webrtc widget