Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–12 of 12 · Most stars
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
Translated【A common used C++ & Python DAG framework】 A general-purpose, third-party-dependency-free, cross-platform parallel computing framework based on flow graphs, listed in awesome-cpp. Welcome to star & fork & discuss
Simple but powerful DAG scheduler and dashboard
- directed-acyclic-graph
- airflow
- dashboard
- go
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
Daggy - Data Aggregation Utility and C/C++ developer library for data streams catching
- serverless-framework
- streaming
- stream-processing
- qt
Distributed batch data processing framework
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
Distributed, Scalable, Fault-Tolerant, Minimalistic Workflow Engine - No DAGs, No YAML, No Cumbersome Diagrams, Just Code
- business-process
- kafka
- kafka-streams
- micorservices
ada privilege escalation