Open-Source Agent Observability & Evals for AI Platform teams. Trace, test, route and govern every agent & coding assistant in the company.
Repository Index
47,133 repositories indexed
1–6 of 6 · Most stars
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
- ai-agents
- benchmark
- coding-agents
- embodied-ai
Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.
- bland
- dspy
- livekit
- llm
Self-hosted AI-assisted CTI-to-detection workbench for ATT&CK mapping, IOC/CVE intelligence, Threat Radar, malware triage, Attack Simulation, and SIEM validation.
Open-source simulation harness for testing AI agents. Simulate LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls.
- ai-agents
- ai-testing
- llm
- llm-evaluation