Sign in

Repository Index

132,241 visible repositories in the index

Tags: LLM evaluation
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–19 of 19 · Most stars

Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

  • ai
  • diagnostic-tool
  • agent-evaluation
  • ai-alignment

🧩 FrontierAgent, our agent framework, open-sourced alongside it — native command-line TUI, ReAct and Agent Team modes, one command on macOS and Linux, no preinstall, no hard Docker dependency.

An open-source Harness Engineering platform for coding agents—define harnesses as code, run controlled experiments, inspect evidence, and compare outcomes. Turn task evidence into actionable team and organization insights.

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.

CLI for testing and evaluating AI agent workflows, custom agents, and skills with repeatable scenarios, graders, evidence, and CI integration.

Stop your AI from making things up — it proposes, deterministic tools decide, every claim checked against ground truth with evidence. Grounded facts and context survive resets. Reverse engineering is the proving ground. MCP server + CLI.

[JMLR 2026] UQLM: A Python package for LLM hallucination detection using uncertainty quantification

  • ai-evaluation
  • ai-safety
  • hallucination
  • hallucination-detection

FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.

  • accuracy-decline
  • ai-safety
  • audit-trail
  • fact-checking

VisSearch-Bench is a benchmark of 204 human-written tasks that test how well multimodal agents find, organize, and use visual evidence across multiple steps of web search.

DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

  • benchmark
  • calibration
  • decision-making
  • evaluation