Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
95,842 visible repositories in the index
1–24 of 44 · Most stars
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Apache Spark - A unified analytics engine for large-scale data processing
Building event-driven applications the easy way in Go.
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
Fast web applications through dynamic, partially-stateful dataflow
Canonical actor model implementation for .NET with local + distributed actors in C# and F#.
A network event stream processing system, in Clojure.
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
High-performance time-series aggregation for PostgreSQL
Goka is a compact yet powerful distributed stream processing library for Apache Kafka written in Go.
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
YTsaurus is a scalable and fault-tolerant open-source big data platform.
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
Ruby and Rails efficient Kafka processing framework
Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.
The Feldera Incremental Computation Engine
Lightweight data stream processing engine for IoT edge
👜 Tiny and fast reactive/iterable programming library
Probabilistic data structures for processing continuous, unbounded streams.
Python Streaming DataFrames for Kafka
A platform that makes it easy for developers to build realtime, cost-effective, operations-focused applications