Repository Index

LLM inference in C/C++

• ggml

Distribute and run LLMs with a single file.

• cross-platform
• gguf
• llama-cpp
• local-ai

Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.

• onednn
• oneapi
• performance
• cpp

TranslatedUsing dense-compute cards to accelerate sparse, large-VRAM hosts, where the hosts include AI Max+ 395, M3 512G, DGX Spark, and others.

• llama-cpp
• pipeline-parallelism
• prefill-decode
• rtx-3060

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

• homebrew
• llama-cpp
• llm
• playstation-5

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• apple-silicon
• cuda
• gguf
• inference

Fast fork of llama.cpp designed for Qwen3.8-Flash-Next

llama.cpp fork that abliterates at inference

• abliteration
• ai
• inference
• llama-cpp