Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
Repository Index
47,133 repositories indexed
1–5 of 5 · Most stars
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
CLI task management & automation tool
- python
- build-tool
- build-automation
- task-runner
Build and deploy a serverless data pipeline on AWS with no effort.
- aws-cdk
- data-pipeline
- stepfunctions
- glue