A hierarchical inference engine for frontier MoE models: hot experts on the GPU (CUDA, Vulkan), warm ones in RAM, cold ones streamed from NVMe. Written in C, runs on the hardware you already own.
Repository Index
111,821 visible repositories in the index
1–20 of 20 · Most stars
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
- amd
- llama
- llm
- llm-inference
An all-in-one, pure C++ inference engine for audio models, powered by ggml. Supports TTS, STT, VAD, voice conversion, music generation, and more, with highly optimized performance. No Python dependency.
- ai
- amd-gpu
- apple-silicon
- asr
The Open Source Alternative to Cluely - A lightning-fast, privacy-first AI assistant that works seamlessly during meetings, interviews, and conversations without anyone knowing. Built with Tauri for native performance, just 10MB. Completely undetectable in video calls, screen shares, and recordings.
- ai-assistant
- claude
- cluely-alternative
- desktop-app
LLM Inference Engine for Metal, CUDA and Vulkan.
Run AI models locally on your machine with node.js bindings for llama.cpp. Enforce a JSON schema on the model output on the generation level
Hypernetworks that adapt LLMs for specific benchmark tasks using only textual task description as the input
- fine-tuning
- hypernetworks
- llm
- lora
TranslatedA multi-account gateway that turns Tencent WorkBuddy accounts into an OpenAI-compatible API, while automatically completing all tasks in the task center. Includes a web management panel (account pool visualization / points tasks / hot configuration updates). An enhanced fork based on Sliverkiss/workbuddy2api
A Self-adaptation Framework🐙 that adapts LLMs for unseen tasks in real-time!
🚀 A real-time monitoring dashboard for Claude Code & Codex, built with SQLite3, Node.js, Express, React, Vite, TailwindCSS, & WebSockets. It tracks sessions, agent activity, tool usage, and subagent orchestration, providing live analytics, a Kanban status board, status notifications, a cute buddy, & an interactive web UI/MacOS/Windows native app.
- claude-agents
- claude-code
- claude-skills
- express
Watch a Claude Code or Codex session as a live flow graph, in your terminal or your browser.
- agent-visualization
- claude-code
- flow
- codex
RAG Search API
- ai-search-engine
- rag
a self-hosted webui for 30+ generative ai
- bark
- controlnet
- gfpgan
- gradio
Benchmarking Generalized Out-of-Distribution Detection
- out-of-distribution-detection
- anomaly-detection
- open-set-recognition
- novelty-detection
Real-time web cockpit for OpenClaw: voice conversations, agent automated kanban board, workspace/file control, sub-agent sessions, inline charts, and usage visibility.
Hypernetworks that update LLMs to remember factual information
37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- avx2
- c
- cpu-inference
- gguf
Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
BulkFormer: A large-scale foundation model for human bulk transcriptomes