Repository Index

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

TranslatedOpen-source AI API relay benchmark, rankings & switcher for Claude Code / Codex: measured availability, latency and price

• ai
• api-relay
• api-relay-pricing
• benchmark

This repository introduces PIXIU, an open-source resource featuring the first financial large language models (LLMs), instruction tuning data, and evaluation benchmarks to holistically assess financial LLMs. Our goal is to continually push forward the open-source development of financial artificial intelligence (AI).

• aifinance
• fintech
• llama
• machine-learning

A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.

• ai-agents
• benchmarking
• llm-evaluation
• python

How many bugs can LMs find & fix in large codebases?

• ai
• ai-agents
• benchmark
• benchmarking

Evolved DSH plugin harnesses for five agent benchmarks, with versioned artifacts, evolution curves and Life-Harness comparisons

• agent-benchmark
• dsh
• harness
• leaderboard