LLM inference in C/C++
- ggml
Repository Index
47,133 repositories indexed
1–9 of 9 · Most stars
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.
- onednn
- oneapi
- performance
- cpp
TranslatedUsing dense-compute cards to accelerate sparse, large-VRAM hosts, where the hosts include AI Max+ 395, M3 512G, DGX Spark, and others.
- llama-cpp
- pipeline-parallelism
- prefill-decode
- rtx-3060
llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8
Official library for RaBitQ vector quantization and approximate nearest-neighbor search, in C++ and Python.
- nearest-neighbor-search
- quantization
- vector-database
- vector-search
Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.
Fast fork of llama.cpp designed for Qwen3.8-Flash-Next
llama.cpp fork that abliterates at inference