🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
Repository Index
47,133 repositories indexed
1–11 of 11 · Most stars
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Evaluation code for various unsupervised automated metrics for Natural Language Generation.
- natural-language-generation
- natural-language-processing
- nlg
- nlp
Build AI text detectors whose decisions are mechanistically grounded and human-auditable (EMNLP 2026)
- ai-text-detection
- explainable-ai
- interpretability
- mechanistic-interpretability
An agent that compiles git maintenance chores into reusable programs, then decides whether to replay them by running checks rather than by guessing the task looks familiar
- agent-architecture
- git-automation
- llm-agents
- program-synthesis
:sunrise: A comprehensive, cutting-edge open-source framework for next-generation Transformers, with compute-matched benchmarking and a platform for autonomous architecture research.
- auto-research
- benchmark
- deep-learning
- language-models
TavernGym and TavernBench: a faithful, full-content Hearthstone Battlegrounds simulator with an RL interface, and a 960-task decision benchmark. Code and data for "Build, Don't Play".
- game-ai
- hearthstone
- reinforcement-learning
- rust
Runtime defense against memory poisoning in AI agents (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.
Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.
- bland
- dspy
- livekit
- llm
Catch your AI agents when they lie about what they shipped — verifies claims against git instead of believing the agent.
- agent-orchestration
- cli
- developer-tools
- llm-agents
The E2E AI testing tool | No ML Overhead