🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
- machine-learning
- ai
- ml
- mlflow
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
Open Source LLM Gateway 🦚
- openclaw
- cost-tracking
- llm-observability
- token-tracking
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
Open-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces.
- llm-observability
- agent-monitoring
- agent-observability
- ai-error-monitor
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
Track AI API quotas across Synthetic, Z.ai, Anthropic (Claude Code), Codex, GitHub Copilot & Antigravity in real time. Lightweight background daemon (<50MB RAM), SQLite storage, Material Design 3 dashboard. Zero telemetry.
- developer-tools
- golang
- quota-tracker
- ai-api-monitoring
Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request.
- ai-gateway
- anthropic
- claude-code
- codex
Tamper-evident tracing, policy gates and cross-checks for AI agents: see what the agent did, and why.
- agent-security
- ai-agents
- audit-log
- claude-code
Fast, cross-platform, real-time token usage tracker and cost monitor for Claude Code / Codex CLI / Antigravity CLI / Qwen Code / Cline / Zoo Code / Kilo Code / GitHub Copilot / OpenCode / Pi Agent / Piebald.
- analyzer
- claude-code
- codex
- usage
Ultra-fast token & cost tracker for LLM Token Usage (e.g. Claude Code)
TranslatedLocal-first Rust TUI/CLI for auditing AI coding-agent sessions: cost, tokens, latency, failures, and health
See what your AI coding agent is doing. Multi-provider assistant & session monitor for VS Code and the terminal — inline completions, code transforms, and a full TUI dashboard tracking tokens, cost, and context across Claude Code, OpenCode, and Codex. Switch between multiple Claude Code and Codex accounts.
- copilot-alternative
- inline-completion
- claude-code-cli
- code-reviews
TranslatedOpen-source unified usage dashboard for Claude — track tokens, quota, costs across Claude Code, claude.ai & API in one panel. Suitable for Pro/Max subscribers and developers. https://dyp23yngrtumg.ok.kimi.link
OpenTelemetry-native tracing, token tracking, cost attribution, and prompt analytics for Go backends integrating LLMs.
Local-only observability for AI agents on your machine. One timeline across coding and non-coding agents.
- ai-agents
- claude-code
- cli
- cursor