Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

• database
• stream-processing
• rust
• postgresql

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time

Ultra-high performance reactive programming

Kubernetes-native platform to run massively parallel data/streaming jobs

• kubernetes
• stream-processing
• data-processing
• k8s

High-performance time-series aggregation for PostgreSQL

• realtime
• stream-processing
• analytics
• sql

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

• clickhouse
• kakfa
• sql
• redpanda

A lightweight stream processing library for Go

• stream-processing
• pipeline
• etl
• kafka

Lightweight data stream processing engine for IoT edge

• edge
• sql
• iot
• edge-analytics

👜 Tiny and fast reactive/iterable programming library

• callbacks
• reactive
• iterables
• observables

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊

• change-data-capture
• data
• data-collection
• data-engineering

Tenzir is the data pipeline engine for security teams.

• incident-response
• threathunting
• siem
• soc

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

• iot
• analytics
• edge
• stream-processing

🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications

• event-driven
• generics
• go
• golang

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

• go
• data-integration
• etl
• data-pipeline

Compositional pipelines

Streaming reactive and dataflow graphs in Python

• python
• python3
• stream
• data-pipeline

Series and Panels for Real-time and Exploratory Analysis of Data Streams

• time-series
• real-time
• data-stream
• series-manipulation

Machine is a workflow/pipeline library for processing data

• golang
• golang-library
• golang-package
• workflow