An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
Repository Index
93,846 visible repositories in the index
1–13 of 13 · Most stars
🪢 Open source AI engineering platform: AI Gateway, agent observability, evals, and prompt/skill management, built on OpenTelemetry. One open platform for every AI call and every agent. Self-host or use Langfuse Cloud.
- analytics
- llm
- llmops
- large-language-models
A framework for few-shot evaluation of language models.
- evaluation-framework
- language-model
- transformer
Translated非线智能 NoneLinear (Non-linear Intelligence) - ReLE evaluation: capability evaluation of Chinese AI large models (continuously updated): it currently covers 374 large models, including commercial models such as chatgpt, gpt-5.4, Google gemini-3.1-pro, Claude-4.6, 文心 ERNIE-X1.1, ERNIE-5.0, qwen3.6-max, qwen3.6-plus, Baichuan, iFlytek Spark, and SenseTime senseChat, as well as open-source large models such as step3.5-flash, kimi-k2.6, ernie4.5, MiniMax-M2.7, deepseek-v4, Qwen3.6, llama4, Zhipu GLM-5.1, MiMo-V2, LongCat, gemma4, and mistral. It provides not only leaderboards but also a large model defect library with more than 2,000,000 entries! This makes it convenient for the broad community to research, analyze, and improve large models.
- agentic-ai
- artificial-intelligence
- llm-agent
- llm-evaluation
The open source research environment for AI researchers to seamlessly train, evaluate, and scale models from local hardware to GPU clusters.
Align Anything: Training All-modality Model with Feedback
- large-language-models
- multimodal
- rlhf
- chameleon
Same agent, Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost.
- observability
- llmops
- ai-observability
- openai
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
A desktop app to prototype agent ideas, inspect every harness step, replay failures, and evaluate performance, all in one place. Local-first, cloud-ready for managed agents.
- agent
- deer-flow
- develop
- electrobun
Benchmark for tracking model capability after release.
Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.
- ui
- responsible-ai
- data-science
- fairness
A unified framework for building, evaluating, and recursively improving embodied agents. 🚀 Code release coming soon.
DecisionPivot: a diagnostic benchmark for decision models in LLM systems. Contrast pairs that differ in one decisive detail, generators with independent solvers, scorer and runners.
- benchmark
- calibration
- decision-making
- evaluation