Build tested agent skills and govern their lifecycle through a user-defined marketplace: evidence, discovery, updates, rollback, quarantine, and 17-platform distribution.
Repository Index
77,353 visible repositories in the index
121–144 of 180 · Most stars
Doing simple retrieval from LLM models at various context lengths to measure accuracy
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
- machine-learning
- experimentation
- llm-prompting
- llm-test
拼好RAG：手搓并融合了GraphRAG、LightRAG、Neo4j-llm-graph-builder进行知识图谱构建以及搜索；整合DeepSearch技术实现私域RAG的推理；自制针对GraphRAG的评估框架| Integrate GraphRAG, LightRAG, and Neo4j-llm-graph-builder for knowledge graph construction and search. Combine DeepSearch for private RAG reasoning. Create a custom evaluation framework for GraphRAG.
- deepresearch
- deepsearch
- evaluation
- graphrag
A Heterogeneous Benchmark for Information Retrieval. Easy to use, evaluate your models across 15+ diverse IR datasets.
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
- mcp
- modelcontextprotocol
- mcp-clients
- mcp-tools
The Fable Workflow: how Claude Fable 5 worked, distilled into skills any model can run, with the eval that keeps it honest. Think / act / prove.
Local typed decisions, contrastive data curation, and model evaluation.
EdegQuake 🌋 High-performance GraphRAG inspired from LightRag written in Rust; Transform documents into intelligent knowledge graphs for superior retrieval and generation
- graphrag
- knowledge-graph
- lightrag
- rag
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
[CVPR 2025] A Comprehensive Benchmark for Document Parsing and Evaluation
A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
- agent
- deer-flow
- develop
- electrobun
:cloud: :rocket: :bar_chart: :chart_with_upwards_trend: Evaluating state of the art in AI
🤖 A Python library for learning and evaluating knowledge graph embeddings
- knowledge-graph-embeddings
- knowledge-graphs
- machine-learning
- link-prediction
[ACL2026] "MiniRAG: Making RAG Simpler with Small and Open-Sourced Language Models"
An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
- deep-learning
- evaluation
- foundation-models
- instruction-following
[ACL 2023] One Embedder, Any Task: Instruction-Finetuned Text Embeddings
- embeddings
- information-retrieval
- language-model
- text-classification
NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.
CORTEX RAG is an enterprise retrieval and knowledge assistant that helps teams find accurate answers from company data with citations, permission-aware retrieval, and fast deployment
SkillsBench evaluates how well skills work and how effective agents are at using them.
- benchmark
- skillsbench
Official github repo for C-Eval, a Chinese evaluation suite for foundation models [NeurIPS 2023]
EVA OS — A real-time multimodal AIOS for next-generation hardware, enabling your devices being “alive” and as intelligent as a real brain.
- aios
- multimodal-ai
- real-time
- robotics
[CVPR2024 Highlight] VBench - We Evaluate Video Generation
- aigc
- evaluation-kit
- gen-ai
- stable-diffusion
Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024