Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Repository Index
47,133 repositories indexed
1–24 of 25 · Most stars
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
Painless Java in Neovim
- autocompletion
- debugging
- diagnostics
- language-server-client
Elegant Lua unit testing.
- lua
- testing
A utility to safely generate malicious network traffic patterns and evaluate controls.
✈️ 自托管的多渠道机票价格监控：航班明细采集 → 钉钉图文推送 → 达标四层强提醒。国内航线与国际航线（五洲目的地）同一套采集-判定-推送链路；数据留在本机，配置全在网页，开源免费。
- crawler
- dingtalk
- flight
- playwright
Assertion library for Lua
Power Assert in Elixir. Shows evaluation results each expression.
- power-assert
- testing-tools
- assertion-library
- assert
A Neovim plugin to easily run and debug Jest tests
A Neovim wrapper for running tests
Lightweight Lua test framework
Run your tests inside nvim in split window or popup with live feedback
- go
- golang
- lua
- neovim
Easy NMOS setup an testing using Docker compose/available containers
xUnit-style + randomized unit testing framework for Lua (and C projects using Lua, etc.)
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
- agent-evaluation
- evaluation-framework
- evaluation-metrics
- java
GitHub workflow for automatically generating Luarocks releases from tags and running busted tests
- automation
- busted
- cicd
- luarocks
Go Coverage for Neovim
Artifact evaluation of paper: MorFuzz: Fuzzing Processor via Runtime Instruction Morphing enhanced Synchronizable Co-simulation
Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.
- bland
- dspy
- livekit
- llm
⚡ JEval helps you to evaluate your JMeter test plan and provides recommendation before you start your performance testing. All contributions welcome 🙏.
- jmeter
- python
- performance
- performance-testing
Neovim plugin for displaying code coverage
The E2E AI testing tool | No ML Overhead
Neotest adapter for running busted tests using neovim as a lua interpreter
- busted
- neotest
- neovim
- neovim-plugin
Open-source simulation harness for testing AI agents. Simulate LLM and tool calls to test edge cases, failure paths, and agent logic without live API calls.
- ai-agents
- ai-testing
- llm
- llm-evaluation