The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Repository Index
138,199 visible repositories in the index
1–24 of 36 · Most stars
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
- machine-learning
- ai
- ml
- mlflow
Supercharge Your LLM Application Evaluations 🚀
- llm
- llmops
- evaluation
Python SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view
- agentneo
- agents
- ai-performance-optimization
- llm-testing
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
BISHENG is an open-source enterprise Agent platform for building, deploying, and managing AI applications. It combines AI agents, workflows, RAG and knowledge management, desktop agent integration, and vibecoding app hosting with unified model management, evaluation, observability, access control, and enterprise administration.
AI Observability & Evaluation
- llmops
- ai-monitoring
- ai-observability
- llm-eval
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.
- python
- fine-tuning
- gguf
- huggingface
Open-source observability for your GenAI or LLM application, based on OpenTelemetry
- llmops
- observability
- open-telemetry
- metrics
🐢 Open-Source Evaluation & Testing library for LLM Agents
- mlops
- ml-validation
- ml-testing
- ai-testing
Evaluation and Tracking for LLM Experiments and AI Agents
- machine-learning
- neural-networks
- explainable-ml
- llmops
Community maintained hardware plugin for vLLM on Huawei Ascend
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
- machine-learning
- experimentation
- llm-prompting
- llm-test
🤖Self-Modifying Framework from the Future 🔮 World's First AMS
No-code multi-agent framework to build LLM Agents, workflows and applications with your data
- ai
- generative-ai
- llm-chain
- llm-framework
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
A deep learning toolkit for healthcare AI: EHR (MIMIC, eICU, OMOP), clinical text, signals and imaging. 30+ models, reproducible pipelines.
- healthcare
- data-mining
- deep-learning
- preprocessing
[GenAI Application Development Framework] 🚀 Build GenAI application quick and easy 💬 Easy to interact with GenAI agent in code using structure data and chained-calls syntax 🧩 Use Event-Driven Flow *TriggerFlow* to manage complex GenAI working logic 🔀 Switch to any model without rewrite application code
- agent
- agent-framework
- python
- framework
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
OpenTelemetry Instrumentation for AI Observability
A framework for comprehensive diagnosis and optimization of agents using simulated, realistic synthetic interactions
- agent
- evaluation
- llmops
- simulator
AuditPilot: auditable enterprise AI agents for evidence-grounded workflows, governed tools, evaluation harnesses, human review, and remediation delivery.
- agent-runtime
- agentic-rag
- ai-agent
- audit
DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models. 🤖💤
Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.
- ai-agents
- ci-cd
- evals
- llm-evaluation