llama.cpp fork with additional SOTA quants and improved performance
Repository Index
111,085 visible repositories in the index
1–2 of 2 · Most stars
KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM
- ggml
- kv-cache
- llama-cpp
- llm-inference