Framework for evaluating and improving agents
- evals
- rl-environments
- terminal-bench
132,241 visible repositories in the index
1–12 of 12 · Most stars
Framework for evaluating and improving agents
🧩 FrontierAgent, our agent framework, open-sourced alongside it — native command-line TUI, ReAct and Agent Team modes, one command on macOS and Linux, no preinstall, no hard Docker dependency.
A benchmark for LLMs on complicated tasks in the terminal
CORTEX RAG is an enterprise retrieval and knowledge assistant that helps teams find accurate answers from company data with citations, permission-aware retrieval, and fast deployment
Stop your AI from making things up — it proposes, deterministic tools decide, every claim checked against ground truth with evidence. Grounded facts and context survive resets. Reverse engineering is the proving ground. MCP server + CLI.
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025)
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
Measuring and evolving with the frontier of agent work
[EMNLP 2024 Demo] MarkLLM: An Open-Source Toolkit for LLM Watermarking
Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains
Code and data for the paper "Evaluating German Transformer Language Models with Syntactic Agreement Tests" (Zaczynska et al., 2020)