Repository Index

LLM inference in C/C++

• ggml

Distribute and run LLMs with a single file.

• cross-platform
• gguf
• llama-cpp
• local-ai

The AI-native database built for LLM applications, providing incredibly fast hybrid search of dense vector, sparse vector, tensor (multi-vector), and full-text.

• ai-native
• nearest-neighbor-search
• rag
• vector-search

Open-source library of optimized deep learning operations (matmul, convolution, attention) for CPUs (x64, AArch64, RISC-V) and Intel GPUs. Used by PyTorch, TensorFlow, OpenVINO, and ONNX Runtime.

• onednn
• oneapi
• performance
• cpp

使用稠密算力卡对稀疏大显存主机进行加速，主机包括 AI Max+ 395、M3 512G、DGX Spark 等。

• llama-cpp
• pipeline-parallelism
• prefill-decode
• rtx-3060

Embeddable, in-memory, document-oriented database with a high-level Query builder interface.

• database
• embedable
• go
• in-memory

llama.cpp on a jailbroken PS5: run any GGUF model natively on the console, starting with Qwen 3.8

• homebrew
• llama-cpp
• llm
• playstation-5

Official library for RaBitQ vector quantization and approximate nearest-neighbor search, in C++ and Python.

• nearest-neighbor-search
• quantization
• vector-database
• vector-search

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

• apple-silicon
• cuda
• gguf
• inference

Fast fork of llama.cpp designed for Qwen3.8-Flash-Next

llama.cpp fork that abliterates at inference

• abliteration
• ai
• inference
• llama-cpp