A high-throughput and memory-efficient inference and serving engine for LLMs
Repository Index
47,133 repositories indexed
1–24 of 88 · Most stars
✨ Zero-config AI chat assistant. No API key needed — sign up and instantly chat with GPT-5, Claude 4, Gemini 2.5, DeepSeek & 100+ top models. Pay-as-you-go saves you more. Available on Web, iOS, macOS, Android, Linux, Windows.
- nextjs
- vercel
- webui
- cross-platform
Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.
A no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.
Wrap Antigravity, ChatGPT Codex, Claude Code, Grok Build, Muse Code, Devin as an OpenAI/Gemini/Claude/Codex compatible API service, allowing you to enjoy the free Gemini Series, GPT Series, Grok Series, Claude model through API
- claude-code
- cluade
- gemini
- openai
Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.
🦍 The API and AI Gateway
- api-gateway
- microservices
- api-management
- serverless
An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
SGLang is a high-performance serving framework for large language models and multimodal models.
🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
- machine-learning
- ai
- ml
- mlflow
Agent2Agent (A2A) is an open protocol enabling communication and interoperability between opaque agentic applications.
- a2a
- a2a-mcp
- a2a-protocol
- a2a-server
Self-Hosting Guide. Learn all about locally hosting (on premises & private web servers) and managing software applications by yourself or your organization. Including Cloud, LLMs, WireGuard, Automation, Home Assistant, and Networking.
LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Translated🔥 MaxKB is an open-source platform for building enterprise-grade agents. A powerful and easy-to-use open-source enterprise-grade agent platform.
Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
The Cloud-Native API Gateway and AI Gateway
- cloud-native
- nginx
- api-management
- luajit
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
FauxPilot - an open-source alternative to GitHub Copilot server
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
- gateway
- generative-ai
- llmops
- llms