Repository Index

Official Repository of paper OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning

From a universe of models to Collective Model Intelligence.

• collective-model-intelligence
• model-capabilities
• model-collaboration

How many bugs can LMs find & fix in large codebases?

• ai
• ai-agents
• benchmark
• benchmarking

This is the official repo for paper InteractiveWorld A Verifiable Human-Centric Environment for Building Better Interactive MLLM Assistants

The repository host gives no description.

Runtime defense against memory poisoning in AI agents (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.

• ai-agents
• langchain
• llm-security
• memory-poisoning

DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.

• benchmark
• calibration
• decision-making
• evaluation

DiscoveryRSI: Recursively Learning to Discover Valuable Hypotheses

• agent-framework
• agentic-workflow
• aiforscience
• llm-agents

Investor’s Last Exam: Can AI Agents Do Equity Investment Without Seeing the Future?

A Long-Horizon Benchmark for Self-Improving Inference Systems

Fault-injected tool environments with side-effect grading for evaluating and post-training LLM agents

• agent-evaluation
• ai-agents
• fastapi
• ollama

Evolved DSH plugin harnesses for five agent benchmarks, with versioned artifacts, evolution curves and Life-Harness comparisons

• agent-benchmark
• dsh
• harness
• leaderboard

bench mark for multi world model

LLM and agent evaluation for Java & Kotlin. Runs in JUnit and CI. Spring AI, LangChain4j, Koog, Embabel, and any LLM client.

• agent-evaluation
• evaluation-framework
• evaluation-metrics
• java

Test harness for voice agents. Import from Retell, VAPI, Bland, LiveKit. Run autonomous simulations. Evaluate with LLM judges.

• bland
• dspy
• livekit
• llm

Catch your AI agents when they lie about what they shipped — verifies claims against git instead of believing the agent.

• agent-orchestration
• cli
• developer-tools
• llm-agents

The E2E AI testing tool | No ML Overhead

• ai
• rag
• rag-evaluation
• rag-pipeline

Ideas, tricks, materials with puzzles for AIs and some wetware