Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–24 of 122 · Most stars
Cross-platform, customizable ML solutions for live and streaming media.
- mediapipe
- c-plus-plus
- computer-vision
- deep-learning
Apache Kafka - A distributed event streaming platform
Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.
- rocketmq
- cloud-native
- eventing
- java
Apache Pulsar - distributed pub-sub messaging system
Logstash - transport and process your logs, events, or other data
Open source RabbitMQ: core server and tier 1 (built-in) plugins
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
ZeroMQ core engine in C++, implements ZMTP/3.1
- zeromq
- zmq
- messaging
- concurrency
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
Building event-driven applications the easy way in Go.
- watermill
- event-driven
- event-sourcing
- sagas
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Efficient reliable UDP unicast, UDP multicast, and IPC message transport
- messaging
- java
- c-plus-plus
- ipc
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
Apache Storm
- storm
- apache
- distributed
- streaming
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Upserts, Deletes And Incremental Processing on Big Data.
- hudi
- apachehudi
- datalake
- bigdata
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.