The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Repository Index
142,187 visible repositories in the index
1–20 of 20 · Most stars
The Cloud-Native API Gateway and AI Gateway
- cloud-native
- nginx
- api-management
- luajit
Open-source, high-performance AI gateway that works as your LLM gateway, MCP gateway and agent gateway. One OpenAI-compatible API for 10,000+ models, with failover, load balancing, semantic caching, virtual keys, budgets and MCP Code Mode. Works with any SDK or agent and adds under 200µs of overhead at 5k RPS.
- gateway
- llm-gateway
- guardrails
- mcp-gateway
TranslatedSelf-hosted AI gateway for multi-channel, multi-credential setups — API keys and subscription accounts, scheduling, failover, request logs and usage.
TranslatedGemini polling proxy service
- gemini
- gemini-api
- googlesearch
- imagen-3
⚡️ Open-source AI Gateway — Use any SDK to call 100+ LLMs. Built-in failover, load balancing, cost control & end-to-end tracing.
Codex/ChatGPT multiple account load balancer & proxy with usage tracking, dashboard, and OpenCode-compatible endpoints
TranslatedLocal AI routing gateway with Agent client takeover — reuse existing client logins, call 50+ AI clients through one base URL, with unified OpenAI / Anthropic / Images APIs, multi-account routing, check-in, balance and request reports
- autoclaw
- catpaw
- local-proxy
- openai-compatible
The Self-Coding System for Your App — Alan AI SDK for Web
- voice-ai
- conversational-ai
- machine-learning
- ai-chat
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Fullstack SaaS Boilerplate built with tRPC, Fastify and React
OpenAI-compatible API for Gemini Business with multi-account load balancing and multimodal capabilities (image/video generation, file parsing) | 将 Gemini Business 转为 OpenAI 兼容接口,支持多账户负载均衡及多模态能力(图像生成、视频生成、解析文件)
This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI format uniformly, and support load balancing. Currently supported backend services include: OpenAI, Anthropic, DeepBricks, OpenRouter, Gemini, Vertex, etc.
XAI - An eXplainability toolbox for machine learning
在线考试系统
Cost tracking for DeepSeek Harness: session and model costs, token usage, budgets, provider balances, and coding-plan quotas. Bilingual English/Chinese UI.
- cost-tracking
- deepseek
- deepseek-api
- deepseek-harness
A proxy server for multiple ollama instances with Key security
The fastest, lightest, and easiest-to-integrate AI gateway on the market. Fully open-sourced.
The Self-Coding System for Your App — Alan AI SDK for React Native
- react-native
- sdk
- voice
- voice-assistant
Universal LLM Gateway: One API, every LLM. OpenAI/Anthropic-compatible endpoints with multi-provider translation and intelligent load-balancing.