A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Repository Index
111,821 visible repositories in the index
1–24 of 26 · Most stars
DeepSeek 4 Flash and PRO local inference engine for Metal, CUDA and ROCm
Inference Llama 2 in one file of pure C
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
Locally run an Instruction-Tuned Chat-Style LLM
Software in C and data files for the popular GloVe model for distributed word representations, a.k.a. word vectors or embeddings
Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案,结构参考alpaca
- llama
- alpaca
- chinese
- vicuna
Modern, secure, portable, multiprotocol bootloader and boot manager.
- bootloader
- bios
- x86
- x86-64
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
llama and other large language models on iOS and MacOS offline using GGML library.
- ai
- ggml
- gpt-2
- gptneox
Python bindings for the Transformer models implemented in C/C++ using GGML library.
- ai
- llm
- transformers
- ctransformers
Pure C inference of Mistral Voxtral Realtime 4B speech to text model
Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)
- rdma
- hpc
- networking
- infiniband
This tool provides an efficient implementation of the continuous bag-of-words and skip-gram architectures for computing vector representations of words. These representations can be subsequently used in many natural language processing applications and for further research.
Llama 2 Everywhere (L2E)
- cosmopolitan
- multios
- blas
- blis
Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
- asr
- ctranslate2
- huggingface
- whisper
React Native binding of llama.cpp
Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4
A minimalist, terminal-native coding agent written in C.
maxxing the ability to run different LLMs on the smallest devices possible
The most atomic way to train and inference a GPT in pure, dependency-free C
- artificial-intelligence
- c
- c-programming
- deep-learning
The simplest and lowest-cost AI integration solution. If you like this project, please give it a Star~ | 最简单、最低成本的AI接入方案。喜欢本项目的话点个 Star 吧~
Port of OpenAI's Whisper model in C/C++ with xtts and wav2lip
lightweight system-level observability for AI Agents