47,133 repositories indexed
1–13 of 13 · Most stars
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
AI Observability & Evaluation
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
Evaluation code for various unsupervised automated metrics for Natural Language Generation.
FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.
A unified framework for building, evaluating, and recursively improving embodied agents. 🚀 Code release coming soon.
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
A Unified Framework for Efficient RSI Research
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
From a universe of models to Collective Model Intelligence.
DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses
LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.
Catch your AI agents when they lie about what they shipped — verifies claims against git instead of believing the agent.