Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
104,105 visible repositories in the index
1–24 of 73 · Most stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Apache Spark - A unified analytics engine for large-scale data processing
Apache Flink
Building event-driven applications the easy way in Go.
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Fancy stream processing made operationally mundane
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
Apache Beam is a unified programming model for Batch and Streaming data processing.
The Reactive Extensions for .NET
Python Stream Processing
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
Fast web applications through dynamic, partially-stateful dataflow
Reactive Extensions for the Go language.
Canonical actor model implementation for .NET with local + distributed actors in C# and F#.
A network event stream processing system, in Clojure.
A modular implementation of timely dataflow in Rust
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
An implementation of differential dataflow using timely dataflow on Rust.
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
Concurrent and multi-stage data ingestion and data processing with Elixir
High-performance time-series aggregation for PostgreSQL
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.