A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Repository Index
111,085 visible repositories in the index
1–24 of 24 · Most stars
A maintained, feature-rich and performance oriented, neofetch like system information tool.
- fastfetch
- flashfetch
- command-line
- fetch
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
Inference Llama 2 in one file of pure C
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
TranslatedDeveloper internal skills cultivation
Coz: Causal Profiling
- profiler
- optimization
- causal-inference
- performance-analysis
Reflective DLL injection is a library injection technique in which the concept of reflective programming is employed to perform the loading of a library from memory into a host process.
MiniMax H3 inference engine for Mac computers
High-efficiency floating-point neural network inference operators for mobile, server, and Web
- neural-networks
- inference
- inference-optimization
- simd
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
Flux 2 image generation model pure C inference
Pure C inference of Mistral Voxtral Realtime 4B speech to text model
Acceleration package for neural networks on multi-core CPUs
- neural-network
- neural-networks
- convolutional-layers
- inference
Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
- asr
- ctranslate2
- huggingface
- whisper
TinyMaix is a tiny inference library for microcontrollers (TinyML).
[NeurIPS 2020] MCUNet: Tiny Deep Learning on IoT Devices; [NeurIPS 2021] MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning; [NeurIPS 2022] MCUNetV3: On-Device Training Under 256KB Memory
- c
- codegenerator
- cpp
- deep-learning
(Unofficial) Firmware Upgrade for Dyson V6/V7 Vacuum Battery Management System
The most atomic way to train and inference a GPT in pure, dependency-free C
- artificial-intelligence
- c
- c-programming
- deep-learning
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- avx2
- c
- cpu-inference
- gguf
Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app.
Legend of Elya — N64 game with a real 6.36M-parameter ternary transformer on the VR4300 MIPS III CPU. Zelda-style dungeon, AI NPCs, byte-level inference at 1.23 tok/s scalar / 2.19 tok/s on the RSP overlay (measured under ares, never on silicon). Built with libdragon.
CatBoost a fast, scalable, high performance Gradient Boosting on Decision Trees library. Golang using Cgo for blazing fast inference CatBoost Model 🚀
- cgo
- golang
- deep-learning
- gradient-boosting
Flappy Bird clone for the Game Boy.