Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Repository Index
94,845 visible repositories in the index
1–9 of 9 · Most stars
🚀 A curated collection of AI engineering resources, roadmaps, tools, and projects covering LLMs, RAG, AI Agents, MCP, fine-tuning, evaluations, and production AI systems.
- agentic-ai
- ai-agents
- ai-engineering
- ai-projects
Supercharge Your LLM Application Evaluations 🚀
- llm
- llmops
- evaluation
Inspect: A framework for large language model evaluations
Awesome-Jailbreak-on-LLMs is a collection of state-of-the-art, novel, exciting jailbreak methods on LLMs. It contains papers, codes, datasets, evaluations, and analyses.
TEAMMATES is a feedback management tool for education
- java
- teammates
- universities
- students
Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
🍇 GRAPE is a Rust/Python Graph Representation Learning library for Predictions and Evaluations
- graph
- high-performance
- machine-learning
- knowledge-graph
A 4B grounding foundation model for physical intelligence. Unifies detection, referring, pointing, OCR, GUI, and layout grounding, achieving SOTA overall across 34 benchmarks and improving robot generalization and data efficiency in controlled evaluations.
- embodied-ai
- gui-grounding
- object-detection
- ocr