Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
98,836 visible repositories in the index
1–23 of 23 · Most stars
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
An orchestration platform for the development, production, and observation of data assets.
Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code
Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
Build, Manage and Deploy AI/ML Systems
🧙 Build, run, and manage data pipelines for integrating and transforming data.
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
Machine Learning Pipelines for Kubeflow
AI Infra / AI Orchestration / AI Control Plane
A workflow language for data-driven computational pipelines
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
cubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台,算法全链路流程,多租户,算力租赁平台,token中转,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务,VGPU虚拟化,云边端协同,边缘计算,自动化标注平台,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库llmops智能体,AI模型市场,支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等,支持ib/roce/RDMA,信创支持
TFX is an end-to-end platform for deploying production ML pipelines
Easy Machine Learning is a general-purpose dataflow-based system for easing the process of applying machine learning algorithms to real world tasks.
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 427 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
Data workflow tool, like a "Make for data"
Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
Build and deploy a serverless data pipeline on AWS with no effort.