Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
71,372 visible repositories in the index
1–24 of 38 · Most stars
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
A network event stream processing system, in Clojure.
- riemann
- events
- clojure
- monitoring
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Ultra-high performance reactive programming
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
High-performance time-series aggregation for PostgreSQL
- realtime
- stream-processing
- analytics
- sql
Ruby and Rails efficient Kafka processing framework
- karafka-framework
- kafka
- kafka-client
- kafka-clients
Fancy stream processing made operationally mundane. This repository is a fork of the original project before the license was changed.
- benthos
- bento
- connect
- connectors
The Feldera Incremental Computation Engine
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Python Streaming DataFrames for Kafka
A platform that makes it easy for developers to build realtime, cost-effective, operations-focused applications
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
A compatibility layer for event-driven abstractions
- stream-processing
- future
- clojure
- async
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
- change-data-capture
- data
- data-collection
- data-engineering
Tenzir is the data pipeline engine for security teams.
- incident-response
- threathunting
- siem
- soc
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
Series and Panels for Real-time and Exploratory Analysis of Data Streams
- time-series
- real-time
- data-stream
- series-manipulation