Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Apache Flink

• scala
• java
• big-data
• flink

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam

Apache Storm

• storm
• apache
• distributed
• streaming

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

• java
• hazelcast
• in-memory
• big-data

Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.

• rule-engine
• java
• dmn
• hacktoberfest

Ruby and Rails efficient Kafka processing framework

• karafka-framework
• kafka
• kafka-client
• kafka-clients

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.

• flow
• rust
• rust-lang
• stream

Mirror of Apache Samza

• scala
• big-data
• samza

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

• iot
• analytics
• edge
• stream-processing

Lightweight real-time big data streaming engine over Apache Pekko

• akka
• stream-processing
• bigdata
• scala

Low-code tool for automating actions on real time data | Stream processing for the users.

• flink
• flink-kafka
• apache-flink
• gui

.NET Stream Processing Library for Apache Kafka 🚀

• kafka-streams-dotnet
• kafka-streams
• kafka-streams-net
• kafka

A streaming / online query processing / analytics engine based on Apache Storm

Mirror of Apache Samoa (Incubating)

• samoa

Machine is a workflow/pipeline library for processing data

• golang
• golang-library
• golang-package
• workflow