Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
- ai
- diagnostic-tool
- agent-evaluation
- ai-alignment
:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Agent RL framework for LLM agents: multi-turn reinforcement learning with StarPO and reasoning-collapse diagnostics
Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.
Knowledge Graph,Question Answering System,基于知识图谱和向量检索的医疗诊断问答系统
A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions
- agent
- evaluation
- llmops
- simulator
RAGChecker: A Fine-grained Framework For Diagnosing RAG
⚕️GenAI powered multi-agentic medical diagnostics and healthcare research assistance chatbot. 🏥 Designed for healthcare professionals, researchers and patients.
Verbalized Sampling, a training-free prompting strategy to mitigate mode collapse in LLMs by requesting responses with probabilities. Achieves 2-3x diversity improvement while maintaining quality. Model-agnostic framework with CLI/API for creative writing, synthetic data generation, and dialogue simulation.
Open-source end-to-end GEO implementation: status analysis, diagnosis, strategy, tickets, execution, verification
- ai-search
- generative-engine-optimization
- geo
- llm
OpenAnt from Knostic is the leading open source LLM-based vulnerability discovery and remediation product, helping defenders proactively find and patch verified security flaws while minimizing both false positives and negatives. Stage 1 detects. Stage 2 attacks. What survives is real.
One API for all LLMs either Private or Public (Anthropic, Llama V2, GPT 3.5/4, Vertex, GPT4ALL, HuggingFace ...) 🌈🐂 Replace OpenAI GPT with any LLMs in your app with one line.
An LLM Based Diagnosis System (https://arxiv.org/pdf/2312.01454.pdf)
Universal Agent Evaluation Framework (UAEF) is a framework-agnostic evaluation system for AI agents. Invoke any agent and score it on multiple metrics spanning tool calling, response quality, safety, performance, and reasoning. Track experiments against baselines, catch regressions automatically, and get LLM-generated insights.
DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
- benchmark
- calibration
- decision-making
- evaluation
DeepSeek Harness Desktop 插件,让 AI Agent 用自然语言构建、检查和维护可编辑的 SideFX Houdini 工程,支持程序化建模、HDA、场景诊断与渲染。 | A DeepSeek Harness Desktop plugin that lets AI agents build, inspect, and maintain editable SideFX Houdini projects with natural language, including procedural modeling, HDAs, scene diagnostics, and rendering.