Repository Index

Apache Kafka - A distributed event streaming platform

• scala
• kafka
• java
• streaming

Orchestrate everything, everywhere. The open-source control plane for workflows across data, infrastructure, business processes and AI agents. Declarative YAML, event-driven, any language, Git-versioned, scalable to millions of executions.

• orchestration
• data-orchestration
• high-availability
• infrastructure-as-code

TranslatedAlibaba's MySQL binlog incremental subscription & consumption component

Apache Flink

• scala
• java
• big-data
• flink

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

• rocketmq
• cloud-native
• eventing
• java

All-in-one IoT Platform - Device management, data collection, processing and visualization.

• visualization
• iot
• iot-platform
• mqtt

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

• database
• distributed-database
• distributed-sql-database
• sql

TranslatedDataX is the open-source version of Alibaba Cloud DataWorks Data Integration.

Apache Hadoop

• hadoop

Apache Pulsar - distributed pub-sub messaging system

• pulsar
• pubsub
• messaging
• streaming

Logstash - transport and process your logs, events, or other data

• etl-framework
• streaming
• logging
• java

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

• workflow-schedule
• azkaban
• airflow
• task-scheduler

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

• change-data-capture
• kafka-connect
• apache-kafka
• debezium

Open-Source Web UI for Apache Kafka Management

• kafka-ui
• kafka-brokers
• kafka
• kafka-streams

Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.

• kafka
• diskless
• automq
• serverless

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

• data-integration
• high-performance
• offline
• real-time

Apache Iceberg

• iceberg
• apache
• hacktoberfest

Efficient reliable UDP unicast, UDP multicast, and IPC message transport

• messaging
• java
• c-plus-plus
• ipc

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam

Pentaho Data Integration ( ETL ) a.k.a Kettle

Apache Storm

• storm
• apache
• distributed
• streaming

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

• java
• hazelcast
• in-memory
• big-data

Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.

• camel
• integration
• java
• microservices

Upserts, Deletes And Incremental Processing on Big Data.

• hudi
• apachehudi
• datalake
• bigdata