🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Repository Index
47,133 repositories indexed
1–10 of 10 · Most stars
Mastra is the modern TypeScript framework for AI-powered applications and agents.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
A framework for building decision-model harnesses for any kind of task. Your program offers the model a short list of next steps, and the model picks one at a time until it can answer.
- classifiers
- decision-models
- evals
- llm-harness
A Claude skill for directing, building and verifying code-rendered videos (HTML/Canvas → MP4): principles, gated workflow, QA scripts and evals.
Observability suite for production voice agents with evals, RSI !
Joy — a local-first personal assistant. Rust core (loop / memory / tools / graph / evals), TypeScript gateways + dashboard, Python thin SDK.