A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
Repository Index
77,353 visible repositories in the index
1–16 of 16 · Most stars
benchmark tooling that loves you ❤️
Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪
- llm-guardrails
- llm-security
- llm-jailbreaks
- llm-scanner
Provides iteration per second benchmarking for Ruby
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
- ai-agents
- benchmarking
- llm-evaluation
- python
Framework to evaluate peformance of ROS 2
Power Assert in Elixir. Shows evaluation results each expression.
- power-assert
- testing-tools
- assertion-library
- assert
Benchmark harness and data: QEMU emulation vs native Buildx nodes for multi-arch container builds.
Fragscapy is a command-line tool to fuzz network protocols by automating the modification of outgoing network packets. It can run multiple successive tests to determine which options can be used to evade firewalls and IDS.
Create, Read, Update and Delete benchmarks at crude hardware speeds for embedded engines and database servers, the Rust successor to UCSB & YCSB
Artifact evaluation of paper: MorFuzz: Fuzzing Processor via Runtime Instruction Morphing enhanced Synchronizable Co-simulation
⚡ JEval helps you to evaluate your JMeter test plan and provides recommendation before you start your performance testing. All contributions welcome 🙏.
- jmeter
- python
- performance
- performance-testing
The E2E AI testing tool | No ML Overhead
Symfony bundle which is used to create fake API Sandbox response with annotations
- php
- php7
- symfony
- symfony2
Open-source simulation harness for testing AI agents. Simulate LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls.
- ai-agents
- ai-testing
- llm
- llm-evaluation
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.