Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
65,764 visible repositories in the index
1–19 of 19 · Most stars
Apache Flink
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Apache Storm
- storm
- apache
- distributed
- streaming
Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.
- rule-engine
- java
- dmn
- hacktoberfest
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Ruby and Rails efficient Kafka processing framework
- karafka-framework
- kafka
- kafka-client
- kafka-clients
Mirror of Apache Samza
- scala
- big-data
- samza
Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.
Lightweight real-time big data streaming engine over Apache Pekko
- akka
- stream-processing
- bigdata
- scala
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
.NET Stream Processing Library for Apache Kafka 🚀
- kafka-streams-dotnet
- kafka-streams
- kafka-streams-net
- kafka
A streaming / online query processing / analytics engine based on Apache Storm
Mirror of Apache Samoa (Incubating)
- samoa
Machine is a workflow/pipeline library for processing data
- golang
- golang-library
- golang-package
- workflow
Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency
A light-weight package for in-app micro batch processing
- batch
- batch-processing
- event-processing
- go
stream count distinct element estimation