Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
- apple
- cua
- lume
- macos
Repository Index
47,133 repositories indexed
1–13 of 13 · Most stars
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.
The LLM Evaluation Framework
- evaluation-metrics
- evaluation-framework
- llm-evaluation
- llm-evaluation-framework
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
Scikit-learn for Recursive Self-improvement (RSI). A modular framework for recursive harness evolution.
- meta-learning
- reinforcement-learning
- rsi
- ai-agents
A unified framework for building, evaluating, and recursively improving embodied agents. 🚀 Code release coming soon.
A Unified Framework for Efficient RSI Research
- agent
- recursive-self-improvement
How many bugs can LMs find & fix in large codebases?
:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers, with compute-matched benchmarking and a platform for autonomous architecture research.
- auto-research
- benchmark
- deep-learning
- language-models
DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses
- agent-framework
- agentic-workflow
- aiforscience
- llm-agents