Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
Repository Index
47,133 repositories indexed
1–7 of 7 · Most stars
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
Harness self-evolution: update targets, theory, evaluation, and reported evidence. Bilingual reading list and project website.
- agent-evaluation
- agent-harness
- agent-memory
- agent-skills
Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
- agent-evaluation
- agentic-ai
- ai-agents
- awesome
Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
- agent-evaluation
- evaluation-framework
- evaluation-metrics
- java
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.