Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–24 of 26 · Most stars
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Distributed stream processing engine in Rust
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
A lightweight stream processing library for Go
Python Stream Processing. A Faust fork
Probabilistic data structures for processing continuous, unbounded streams.
- bloom-filter
- go
- stable-bloom-filters
- cuckoo-filter
Python Streaming DataFrames for Kafka
tools for working with streams of data
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
- change-data-capture
- data
- data-collection
- data-engineering
Build platforms that flexibly mix SQL, batch, and stream processing paradigms
- golang
- event-sourcing
- stream-processing
- brokers
Tenzir is the data pipeline engine for security teams.
- incident-response
- threathunting
- siem
- soc
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications
- event-driven
- generics
- go
- golang
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
- go
- data-integration
- etl
- data-pipeline
.NET Stream Processing Library for Apache Kafka 🚀
- kafka-streams-dotnet
- kafka-streams
- kafka-streams-net
- kafka
Streaming reactive and dataflow graphs in Python
- python
- python3
- stream
- data-pipeline
Series and Panels for Real-time and Exploratory Analysis of Data Streams
- time-series
- real-time
- data-stream
- series-manipulation