Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Cross-platform, customizable ML solutions for live and streaming media.

• mediapipe
• c-plus-plus
• computer-vision
• deep-learning

Apache Kafka - A distributed event streaming platform

• scala
• kafka
• java
• streaming

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

• rocketmq
• cloud-native
• eventing
• java

Apache Pulsar - distributed pub-sub messaging system

• pulsar
• pubsub
• messaging
• streaming

Logstash - transport and process your logs, events, or other data

• etl-framework
• streaming
• logging
• java

Open source RabbitMQ: core server and tier 1 (built-in) plugins

• rabbitmq
• amqp
• messaging
• amqp-0-9-1

Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.

• change-data-capture
• kafka-connect
• apache-kafka
• debezium

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

• kafka
• cpp
• kubernetes
• go

Open-Source Web UI for Apache Kafka Management

• kafka-ui
• kafka-brokers
• kafka
• kafka-streams

Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.

• websocket
• redis
• real-time
• messaging

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

• data-integration
• high-performance
• offline
• real-time

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql

Fancy stream processing made operationally mundane

• message-queue
• stream-processing
• streaming-data
• message-bus

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam

Apache Storm

• storm
• apache
• distributed
• streaming

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

• rust
• database
• sql
• streaming

🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications

• api
• web-api
• server-sent-events
• subscriptions

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time

Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed

• messaging
• quic
• rust
• streaming

Privacy and Security focused Segment-alternative, in Golang and React

• privacy
• warehouse-management
• data-warehouse
• customer-data

Utils for streaming large files (S3, HDFS, gzip, bz2...)

• python
• s3
• hdfs
• webhdfs

KubeBlocks is a Kubernetes Operator designed to manage a variety of databases and streaming systems, including MySQL, PostgreSQL, MongoDB, Redis, RabbitMQ, RocketMQ, and more, within Kubernetes environments.

• mysql-operator
• mongodb-operator
• postgresql-operator
• rabbitmq-operator

Kubernetes-native platform to run massively parallel data/streaming jobs

• kubernetes
• stream-processing
• data-processing
• k8s