Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
Repository Index
47,133 repositories indexed
1–13 of 13 · Most stars
Easy benchmarking of all publicly accessible implementations of convnets
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
Benchmarks for generative chemistry
Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
AgML is a centralized framework for agricultural machine learning. AgML provides access to public agricultural datasets for common agricultural deep learning tasks, with standard benchmarks and pretrained models, as well the ability to generate synthetic data and annotations.
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
Companion repository for “In-Context Learning for Robots: Methods and Applications” — methods, datasets, and benchmarks organized by taxonomy and release date.
How many bugs can LMs find & fix in large codebases?
Fish Audio Drama 3 text-to-speech on the TensorFold native CUDA engine for DGX Spark (GB10): recipe, engine patches, benchmarks. Built with Fish Audio.
Multi-stage Riemannian flow matching for physically valid molecular docking, with GNINA scoring, PoseBusters filtering, CLI inference, and benchmarks.
Prompt-injection firewall for LLM applications — 33 input detectors, 9 output scanners, federated ed25519-signed threat-intel feed. Apache 2.0, 1040 tests, F1 96.0% with 0% false positives. Docker, GitHub Action, LangChain/LlamaIndex/CrewAI integrations.
Scapy benchmarks