A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Repository Index
142,273 visible repositories in the index
1–9 of 9 · Most stars
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
Modern, secure, portable, multiprotocol bootloader and boot manager.
- bootloader
- bios
- x86
- x86-64
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
llama and other large language models on iOS and MacOS offline using GGML library.
- ai
- ggml
- gpt-2
- gptneox
Python bindings for the Transformer models implemented in C/C++ using GGML library.
- ai
- llm
- transformers
- ctransformers
Llama 2 Everywhere (L2E)
- cosmopolitan
- multios
- blas
- blis
Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
- asr
- ctranslate2
- huggingface
- whisper
A minimal, fast, graphical UEFI boot manager written in C. no external dependencies — just a config file :)
- bootloader
- bootmanager
- ricing
- ricing-linux