Enterprise-grade benchmark evaluating whether AI agents deliver verifiable business outcomes in real business systems — general domain, finance, supply chain, and manufacturing.
- mfg
- scm
- erp
- eval
Repository Index
77,353 visible repositories in the index
169–180 of 180 · Most stars
LLM-powered Text-to-SQL evaluation toolkit: schema ingestion, retrieval eval, and standalone SQL judge/re-scorer.
DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
- benchmark
- calibration
- decision-making
- evaluation
Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
- agent-evaluation
- evaluation-framework
- evaluation-metrics
- java
Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.
- bland
- dspy
- livekit
- llm
CancerFoundation: A single-cell RNA sequencing foundation model to decipher drug resistance in cancer
- cancer
- foundation-model
- single-cell
🧭 Let your coding agent off the leash — not off the rails. Guardrails, a hard budget cap & a self-fixing PR loop for Claude Code / Codex / Gemini. Eval-gated 100/100, you always merge.
The E2E AI testing tool | No ML Overhead
Code and data for the paper "Evaluating German Transformer Language Models with Syntactic Agreement Tests" (Zaczynska et al., 2020)
- bert
- transformers
- syntactic-agreement
- german
Open-source simulation harness for testing AI agents. Simulate LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls.
- ai-agents
- ai-testing
- llm
- llm-evaluation
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.