AI system design guide for engineers building production AI systems and evals.
- agentic-ai
- agentic-workflow
- artificial-intelligence
- aws
Repository Index
47,133 repositories indexed
49–72 of 85 · Most stars
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
WFGY is heading toward WFGY 5.0 Polaris Protocol, a major open-source release for AI reasoning, RAG, agents, and real-world workflows. Includes Problem Map, Global Debug Card, WFGY 4.0, and the CFV Easter Egg.
A free AI assistant and survival library that runs on your own computer, with no internet needed. Shows the source of every answer.
Evaluation code for various unsupervised automated metrics for Natural Language Generation.
- natural-language-generation
- natural-language-processing
- nlg
- nlp
A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence. Built on HealthBench.
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.
- accuracy-decline
- ai-safety
- audit-trail
- fact-checking
This repository introduces PIXIU, an open-source resource featuring the first financial large language models (LLMs), instruction tuning data, and evaluation benchmarks to holistically assess financial LLMs. Our goal is to continually push forward the open-source development of financial artificial intelligence (AI).
- aifinance
- fintech
- llama
- machine-learning
A code repository indexing tool to supercharge your LLM experience.
- neovim-plugin
- retrieval-augmented
- embeddings
- rag
Ontology-Augmented Generation for AI agents.
- agent-skills
- document-parsing
- knowledge-graph
- mineru
🔥 The first-ever interactive streaming video generation benchmark.
- avatar-performance
- interaction-streaming-video
- real-time-video-editing
- streaming-video-generation
A production-grade, high-efficiency Agentic RL framework for frontier LLMs with validated recipes.
GLM 5.3 abliterated or uncensored API access with RouterHub. Browse 35 jailbreak prompt links, compare GLM models, and find SDK examples, coding-tool setup, pricing, and evaluation guides.
- abliterated
- abliterated-models
- ai-api
- ai-jailbreaks
A unified framework for building, evaluating, and recursively improving embodied agents. 🚀 Code release coming soon.
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
Fast fork of llama.cpp designed for Qwen3.8-Flash-Next
The first exam for code agents in the physical world: frozen robot tasks from many benchmark suites, each answered with a program and verified in a separate sandbox. Runs on Harbor.
- ai-agents
- benchmark
- coding-agents
- embodied-ai
Official Repository of paper OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
Feather agent harness, training integrations, and evaluation tools for knowledge work
Survey companion for Computational Character Modeling: role-playing methods, data foundations, and evaluation, organized by the manuscript.
- awesome-list
- large-language-models
- role-playing-agents
- survey
One command downloads a complete offline, no-internet emergency library: local AI (CPU/GPU), document Q&A, full Wikipedia, offline maps for any country/region (default US), survival/repair/medical archives, and public-domain manuals. Asks only where to save.
- claude
- claude-code
- emergency-preparedness
- kiwix
DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
- benchmark
- calibration
- decision-making
- evaluation