OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety.
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
LLM-as-a-Verifier is a general-purpose framework that provides fine-grained feedback for any agent without requiring additional training. It achieves SOTA performance across coding, robotics, and medical agentic benchmarks.
Find, benchmark and install in CLI 170+ FREE coding LLM models across 15+ providers in real time
Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends
- evaluation
- evaluation-framework
- evaluation-metrics
- huggingface
A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Open memory & reliability layer for AI agents + independent judging/verification (NACRE judge, caliber-bench meta-benchmark, preregistered criteria) — by Nautilus Platform
Terminal-Bench-Science: Evaluating AI agents on research workflows across scientific domains
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
Open-source whiteboard AI tutor with animated lessons and interactive playback. Compare Claude, Gemini & DeepSeek across 288 generation runs and 64 blind-reviewed boards.
- ai-teacher
- ai-tutor
- ai-tutoring
- ai-whiteboard
VisSearch-Bench is a benchmark of 204 human-written tasks that test how well multimodal agents find, organize, and use visual evidence across multiple steps of web search.