🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Repository Index
47,133 repositories indexed
1–21 of 21 · Most stars
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
- machine-learning
- ai
- ml
- mlflow
Open source observability platform for logs, metrics, traces, RUM (web, android, ios), Session replay, pipelines, SLO and LLM observability. A sophisticated, simple and highly performant alternative to Datadog, Splunk, and Elasticsearch with 140x lower storage costs and single binary deployment.
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
Open Source LLM Gateway 🦚
- openclaw
- cost-tracking
- llm-observability
- token-tracking
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
🚀 A real-time monitoring dashboard for Claude Code & Codex, built with SQLite3, Node.js, Express, React, Vite, TailwindCSS, & WebSockets. It tracks sessions, agent activity, tool usage, and subagent orchestration, providing live analytics, a Kanban status board, status notifications, a cute buddy, & an interactive web UI/MacOS/Windows native app.
- claude-agents
- claude-code
- claude-skills
- express
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
Track AI API quotas across Synthetic, Z.ai, Anthropic (Claude Code), Codex, GitHub Copilot & Antigravity in real time. Lightweight background daemon (<50MB RAM), SQLite storage, Material Design 3 dashboard. Zero telemetry.
- developer-tools
- golang
- quota-tracker
- ai-api-monitoring
Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request.
- ai-gateway
- anthropic
- claude-code
- codex
A comprehensive observability solution for monitoring Claude Code usage, performance, and costs.
Tamper-evident tracing, policy gates and cross-checks for AI agents: see what the agent did, and why.
- agent-security
- ai-agents
- audit-log
- claude-code
Fast, cross-platform, real-time token usage tracker and cost monitor for Claude Code / Codex CLI / Antigravity CLI / Qwen Code / Cline / Zoo Code / Kilo Code / GitHub Copilot / OpenCode / Pi Agent / Piebald.
- analyzer
- claude-code
- codex
- usage
Ultra-fast token & cost tracker for LLM Token Usage (e.g. Claude Code)
TranslatedLocal-first Rust TUI/CLI for auditing AI coding-agent sessions: cost, tokens, latency, failures, and health
See what your AI coding agent is doing. Multi-provider assistant & session monitor for VS Code and the terminal — inline completions, code transforms, and a full TUI dashboard tracking tokens, cost, and context across Claude Code, OpenCode, and Codex. Switch between multiple Claude Code and Codex accounts.
- copilot-alternative
- inline-completion
- claude-code-cli
- code-reviews
TranslatedOpen-source unified usage dashboard for Claude — track tokens, quota, costs across Claude Code, claude.ai & API in one panel. Suitable for Pro/Max subscribers and developers. https://dyp23yngrtumg.ok.kimi.link
OpenTelemetry-native tracing, token tracking, cost attribution, and prompt analytics for Go backends integrating LLMs.
Claude Code dashboard: see where your limits are headed before you hit them. Calibrated usage forecasts, live rate-limit gauges, session costs, memory browser. Single binary, zero config, plus a VS Code extension.
Local-only observability for AI agents on your machine. One timeline across coding and non-coding agents.
- ai-agents
- claude-code
- cli
- cursor
Native macOS app for tracking Claude Code usage — tokens, cost, rate-limit pacing, per-project breakdowns. SwiftUI + SwiftData.