Sign in

Tags / LLM tools /

LLM evaluation

A tool or benchmark that measures the quality of LLM or agent outputs against tests or datasets.

Repositories

56

Net stars gained, last 7 days

+6,443

Top-level Tag

LLM tools

Net stars gained each day, last 90 days

2026-09-08: +1,763 net stars. Most from Albert-Weasker/niubigeo, +1,447.2026-09-13: +1,680 net stars. Most from langwatch/langwatch, +1,233.2026-09-20: +1,966 net stars. Most from Albert-Weasker/niubigeo, +1,059.2026-09-21: +1,937 net stars. Most from Albert-Weasker/niubigeo, +1,168.2026-10-25: +1,863 net stars. Most from lyramakesmusic/neuron-explorer, +855.

Peak days

  1. 1

    Sep 8, 2026 +1,763 net stars (median day +436)

  2. 2

    Sep 13, 2026 +1,680 net stars (median day +436)

  3. 3

    Sep 20, 2026 +1,966 net stars (median day +436)

  4. 4

    Sep 21, 2026 +1,937 net stars (median day +436)

  5. 5

    Oct 25, 2026 +1,863 net stars (median day +436)

A peak day has at least 2 times the net stars of the median day of this chart, and at least 100 more. For each peak day, the list shows the repositories that gained the most net stars that day. It also shows their GitHub releases of that day or the day before (UTC), and their creation date when it was 14 days or less before. These events occur near the same day. They do not show why the stars came.

Largest

Most stars in total now.

  1. readme: The training and evaluation code for state-of-the-art models (e.g., Vicuna, MT-Bench).

  2. readme: It supports LLM-as-a-judge, Code evaluators, user feedback collection, manual labeling, and custom evaluation pipelines via APIs/SDKs.

  3. readme: This repository also holds Cua Driver for desktop automation, Lume for local VMs, CUA-S1 decision models and Cua Bench for evaluating computer-use agents.

  4. readme: Choose from 50+ built-in metrics and LLM judges, or define your own.

  5. readme: **Test your prompts and models** with automated evaluations

  6. readme: Automate your LLM application evaluation with Datasets and Experiments.

  7. description: Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

  8. readme: It is similar to Pytest but specialized for unit testing LLM apps.

  9. readme: 🎯 Objective Metrics: Evaluate your LLM applications with precision using both LLM-based and traditional metrics.

  10. readme: Create and manage metric evaluation of your RAG application:

Newest Admissions

Entered the index by the rising rule, newest first.

  1. description: A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.

  2. description: Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

  3. readme: **StateTrack-Bench** plants one persistent constraint (numerical, format, role/persona, factual-memory or logical-rule) in a conversation, checks compliance at increasing dialogue depths

  4. readme: **InteractiveBench** builds 120 frozen multi-turn tasks on top of it, where one model plays the assistant and another plays the human.

  5. readme: Investor’s Last Exam (**ILE**) is a buy-side-expert-led benchmark for equity investment research.

  6. readme: A benchmark for an open measurement in skill reuse: which replay decision mis-fires less.

  7. readme: RobustRouteBench contains **35,072 instances** across General Text (63 candidate models), Multimodal (45), and Chinese (10)

  8. readme: A document-parsing benchmark with 16,288 tests on 2,937 pages.

  9. readme: Analyze LLM Outputs** so that you understand which models have which tendencies

  10. readme: A benchmark for reasoning about game decisions and rules.

All 56 in the Repository Index

How a repository gets this Tag

An agent reads the description, the GitHub topics, and the start of the README of each repository. It chooses Tags only from the Tag list in the AwesomeRepos source code. For each Tag, it quotes the text. A code check keeps the Tag only when the quote is in that text. The quote under each repository is that evidence. A Tag does not rank repositories. It does not say that one repository can replace another. To report a wrong Tag, use "Report a problem with this page" on the page of the repository. All numbers on this page count only visible repositories.