Pure-Rust inference engine for Qwen GGUF models (Qwen3 to Qwen3.8, dense and MoE up to 770B+). CUDA and Vulkan (NVIDIA, AMD), hybrid CPU+GPU experts, MTP speculative decoding, OpenAI and Anthropic compatible server. Up to ~1.7x llama.cpp decode on a 12 GB RTX 3060.
Repository Index
132,241 visible repositories in the index
1,297–1,320 of 1,377 · Most stars
Extract knowledge from all information sources using gpt and other language models. Index and make Q&A session with information sources.
CaseCite is an open-source, self-hosted legal RAG caselaw research & document analysis platform.
- legal
- legal-ai
- legal-tech
- legaltech
A ChatGPT, Gemini, Claude, and Ollama(offline) plug-in for Embarcadero RAD Studio IDE XE5 to 12.1
- chatgpt
- chatgpt-api
- delphi
- embarcadero
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
One API for every LLM: 170+ providers through one client, with a Rust core and native bindings for every major language.
9RouterX (9R Next Go Generation): Single-binary Go AI router, LLM gateway & 20-40% token saver. Route Claude Code, Cursor & Codex across 155 providers, 28 OAuth & 248 free models (24 keyless) — OpenAI, Claude, Gemini, DeepSeek, Groq, Ollama & MCP.
- ai-gateway
- ai-proxy
- ai-routing
- api-gateway
Find known secrets in your AI coding agent sessions (Claude Code, Codex, Cursor), then guard the next one with local hooks that trace risky commands to the content that dictated them. Open source, no telemetry.
- agent-security
- ai-agents
- ai-security
- claude-code
Capri — an AI desktop companion for Windows with local/API models, Hugging Face, optional voice, memory and permission-based desktop tools.
- ai-assistant
- desktop-companion
- huggingface
- local-llm
Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.
- cuda
- desktop-app
- gguf
- llm
Go AI SDK, the Go way. One unified API across 21+ providers. Streaming, structured output, MCP support, stdlib only. Go AI SDK for AI applications inspired by Vercel AI SDK.
WorkBuddy2API-Hub - OpenAI-compatible local gateway
A DeepSeek Harness (dsh) plugin — turns dsh web into a SillyTavern-style roleplay frontend: character cards, prompt presets, lorebooks, personas, BM25 long-term memory, world-state deltas, and rollback-able floor operations, all on the dsh agent runtime.
- dsh-plugin
- deepseek-harness
- deepseek-harness-plugin
- sillytavern
Free AI models for every coding tool — a local OpenAI-compatible gateway for Claude Code, Codex, Cursor, Cline and OpenCode. Based on 9router (MIT).
Fast, cross-platform, real-time token usage tracker and cost monitor for Claude Code / Codex CLI / Antigravity CLI / Qwen Code / Cline / Zoo Code / Kilo Code / GitHub Copilot / OpenCode / Pi Agent / Piebald.
- analyzer
- claude-code
- codex
- usage
Uncensored AI API and models — one key for Claude, GPT, Gemini, Grok, DeepSeek, GLM and Kimi, with uncensored and abliterated variants where available. OpenAI- and Anthropic-compatible. Pay per token.
- abliterated-models
- ai-gateway
- anthropic-compatible
- claude-api
TranslatedChinese edition of an AI reference image gallery | Costumes (1,407 realistic sets / 1,320 animation sets), 1,499 face shapes, 1,439 pose line drawings and 1,439 depth maps, 843 anime-style (二次元) entries. Designed for AI painting, character design, and comic-drama storyboards, with support for Chinese search, multi-angle browsing, and image copying and downloading. Open-source static site that preserves the original sources and terms of use.
- ai-art
- ai-comics
- art-reference
- character-design
Cross-platform desktop LLM client for OpenAI ChatGPT, DeepSeek R1, Anthropic Claude and more, with a focus on privacy and security.
- desktop-app
- react
- rust
- tauri
Generate your eslint rule with OpenAI and Vercel Edge Functions.
Runtime defense against memory poisoning in AI agents (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.
On-device Vision Model - a fully offline, free vision pipeline that gives text-only models (e.g. DeepSeek) image understanding and multilingual OCR through an OpenAI-compatible local API. Only tokens are needed: no API key, no subscription, no cloud cost. Recommended for native speakers of Chinese, English, Japanese and Korean.
Deepinfer inspired implemetation on SM80 Ampere architecture. Deepest respect for DeepSeek opensource this work!
Ultra-fast token & cost tracker for LLM Token Usage (e.g. Claude Code)
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.