Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
76,855 visible repositories in the index
1–17 of 17 · Most stars
Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Apache Flink
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Python Stream Processing. A Faust fork
Python Streaming DataFrames for Kafka
Distributed Stream Processing
- stream-processing
- stream-processor
- stream-processing-engine
- pony-language
Real-time stream processing for python
- streaming-data
- python
- async
- real-time
Streaming reactive and dataflow graphs in Python
- python
- python3
- stream
- data-pipeline
Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.
- analytics
- feature-store
- fraud-detection
- python
Scramjet Platform is a distributed runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters without exposing network ports to the public internet.
- nodejs
- stream-processing
- transform-as-a-service
- real-time
Stream Processing Made Easy
- machine-learning
- reactive-programming
- kafka
- reactive-systems
Python port of Scramjet framework