Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–14 of 14 · Most stars
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
🌊 Online machine learning in Python
- incremental-learning
- machine-learning
- python
- online-learning
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
Utils for streaming large files (S3, HDFS, gzip, bz2...)
- python
- s3
- hdfs
- webhdfs
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Python Stream Processing. A Faust fork
Python Streaming DataFrames for Kafka
Real-time stream processing for python
- streaming-data
- python
- async
- real-time
Streaming reactive and dataflow graphs in Python
- python
- python3
- stream
- data-pipeline
A pure Python implementation of Apache Spark's RDD and DStream interfaces.
- data-science
- data-processing
- python
- apache-spark
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
Stream Processing Made Easy
- machine-learning
- reactive-programming
- kafka
- reactive-systems