A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–8 of 8 · Most stars
Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2
- amd
- llm
- strix-halo
- gfx1151
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
A functional JavaScript library that facilitates currying and point-free programming
- ai-friendly
- composition
- curry
- currying
a lightweight ChatGPT plugin for neovim
MicroPython LoRa EBYTE E220 LLCC68 device library complete and tested with Arduino, esp8266, esp32, STM32 and Raspberry Pi Pico (rp2040 boards).
- arduino
- arduino-samd-boards
- e220
- ebyte
MicroPython LoRa EBYTE E22 device library complete and tested with Arduino SAMD, esp8266, esp32, Raspberry, rp2040 and STM32. sx1262/sx1268