Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
- airflow
- apache
- apache-airflow
- python
Repository Index
47,133 repositories indexed
1–24 of 41 · Most stars
Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.
- orchestration
- data-orchestration
- high-availability
- infrastructure-as-code
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
An orchestration platform for the development, production, and observation of data assets.
- data-pipelines
- dagster
- workflow
- data-science
Node-based Visual Programming Toolbox
- photogrammetry
- computer-vision
- 3d-reconstruction
- alicevision
Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
🍴 Web applications made easy. Since 2011.
ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.
- mlops
- machine-learning
- data-science
- production-ready
Blazing-fast, Neuro-symbolic Data-Wrangling toolkit
- csv
- data-wrangling
- opendata
- data-engineering
A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!
- mqtt
- m2m
- scalable
- broker
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
- workflow
- governance
- azkaban
- davinci
The Cloud Native Webhooks Gateway
- webhooks
- convoy
- golang
- automation
Eclipse iceoryx™ - true zero-copy inter-process-communication
- iceoryx
- eclipse
- ipc
- inter-process-communication
Python Stream Processing. A Faust fork
MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilise the resources of multiple clustered machines.
- mongooseim
- erlang
- xmpp
- jabber
visual data prep powered by python
- data
- data-pipelines
- etl
- structured-data
A complete, fully tested and documented data structure library written in pure JavaScript.
- javascript
- data-structures
- collections
- collection
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
Cross-platform beanstalkd queue server admin console.
- aurora
- go
- beanstalkd
- bstk
Automatic data change tracking for PostgreSQL
- audit-log
- audit-trail
- change-data-capture
- change-tracking
Honeynet Project generic authenticated datafeed protocol
- hpfeeds
- python3