Repository Index

LLM inference in C/C++

• ggml

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

• llm-inference
• ai-chat

Official inference framework for 1-bit LLMs

Distribute and run LLMs with a single file.

• cross-platform
• gguf
• llama-cpp
• local-ai

Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.

• onednn
• oneapi
• performance
• cpp

A local inference engine for Apple silicon, built around the model.

• apple-silicon
• coding-agents
• llm-inference
• macos

The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2

• amd
• llm
• strix-halo
• gfx1151

Run large AI models on your own PC: Qwen3.8-Flash-Next on Strata's engine, and GLM 5.3 Flash, DeepSeek V4 Flash, LFM2.5, Qwen3.6, Ornith 1.5, Gemma 4, Tiel Coder and MiniCPM-V on a second engine written for PolyStrata. One-click install for Windows and Linux, OpenAI/Anthropic API on localhost.

• anthropic-api
• cuda
• gemma
• gguf

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

• homebrew
• llama-cpp
• llm
• playstation-5

Xolotl (formerly Strata-DSV4.1): run DeepSeek-V4.1-Flash, all 552B, as a local coding agent on one gaming PC (RTX 4090 + 90 GB RAM) at up to ~27 tok/s, 512K context, October 1017 builds included. Plus GLM-5.3-Flash and Qwen3.8. MoE experts tiered across VRAM, pinned RAM and NVMe; OpenAI + Anthropic API.

• cuda
• deepseek
• deepseek-v4
• gguf

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• apple-silicon
• cuda
• gguf
• inference

A minimalistic C++ Jinja templating engine for LLM chat templates

llama.cpp fork that abliterates at inference

• abliteration
• ai
• inference
• llama-cpp