Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–22 of 22 · Most stars
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
- orchestration
- data-orchestration
- high-availability
- infrastructure-as-code
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Node-based Visual Programming Toolbox
- photogrammetry
- computer-vision
- 3d-reconstruction
- alicevision
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
🍴 Web applications made easy. Since 2011.
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!
- mqtt
- m2m
- scalable
- broker
The Cloud Native Webhooks Gateway
- webhooks
- convoy
- golang
- automation
Eclipse iceoryx™ - true zero-copy inter-process-communication
- iceoryx
- eclipse
- ipc
- inter-process-communication
visual data prep powered by python
- data
- data-pipelines
- etl
- structured-data
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
Automatic data change tracking for PostgreSQL
- audit-log
- audit-trail
- change-data-capture
- change-tracking
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
- data-ops
- data-quality
- data-quality-checks
- data-quality-measurement
Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.
Lightweight library that handles RabbitMQ auto-reconnect and publishing retry routine for you.
- rabbitmq
- amqp
- failover
- go
A blazing fast fully-automated CSV to database importer
- csv2sql
- csvs
- database
- csv-files
Simple MQTT Client.
- mqtt
- client
- mqtt-client
- python
🎲 Dotflow turns an idea into flow! — Lightweight Python library for execution pipelines
- etl
- etl-framework
- etl-pipeline
- python
💬 Lightweight AlgoChat client for corvid-agent — GitHub Pages hosted
- algochat
- algorand
- blockchain
- encrypted-messaging