Repository Index

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

• airflow
• apache
• apache-airflow
• python

Apache Spark - A unified analytics engine for large-scale data processing

• python
• scala
• r
• java

Apache Kafka - A distributed event streaming platform

• scala
• kafka
• java
• streaming

Apache Flink

• scala
• java
• big-data
• flink

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

• rocketmq
• cloud-native
• eventing
• java

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

• database
• distributed-database
• distributed-sql-database
• sql

Apache Hadoop

• hadoop

Apache Pulsar - distributed pub-sub messaging system

• pulsar
• pubsub
• messaging
• streaming

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code

• workflow-schedule
• azkaban
• airflow
• task-scheduler

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

• change-data-capture
• kafka-connect
• apache-kafka
• debezium

Sarama is a Go library for Apache Kafka.

• kafka
• kafka-client
• go

Open-Source Web UI for Apache Kafka Management

• kafka-ui
• kafka-brokers
• kafka
• kafka-streams

CMAK is a tool for managing Apache Kafka clusters

• kafka
• scala
• cluster-management
• big-data

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

• data-integration
• high-performance
• offline
• real-time

Apache Iceberg

• iceberg
• apache
• hacktoberfest

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam

Dockerfile for Apache Kafka

Apache Storm

• storm
• apache
• distributed
• streaming

Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.

• camel
• integration
• java
• microservices

Upserts, Deletes And Incremental Processing on Big Data.

• hudi
• apachehudi
• datalake
• bigdata

Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.

• rule-engine
• java
• dmn
• hacktoberfest

Apache NiFi

• nifi
• java
• hacktoberfest
• apache

Generic command line non-JVM Apache Kafka producer and consumer