A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Repository Index
95,842 visible repositories in the index
1–9 of 9 · Most stars
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
Chinese-Vicuna: A Chinese Instruction-following LLaMA-based Model —— 一个中文低资源的llama+lora方案,结构参考alpaca
- llama
- alpaca
- chinese
- vicuna
Run a 1-billion parameter LLM on a $10 board with 256MB RAM
Unified Communication X (mailing list - https://elist.ornl.gov/mailman/listinfo/ucx-group)
- rdma
- hpc
- networking
- infiniband
This tool provides an efficient implementation of the continuous bag-of-words and skip-gram architectures for computing vector representations of words. These representations can be subsequently used in many natural language processing applications and for further research.
Automatically exported from code.google.com/p/word2vec
maxxing the ability to run different LLMs on the smallest devices possible
Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app.