Repository Index

A curated list of awesome big data frameworks, ressources and other awesomeness.

• awesome
• bigdata
• data-warehouse
• data-analytics

Open-Source Web UI for Apache Kafka Management

• kafka-ui
• kafka-brokers
• kafka
• kafka-streams

Fancy stream processing made operationally mundane

• message-queue
• stream-processing
• streaming-data
• message-bus

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

• rust
• database
• sql
• streaming

🌊 Online machine learning in Python

• incremental-learning
• machine-learning
• python
• online-learning

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time

Utils for streaming large files (S3, HDFS, gzip, bz2...)

• python
• s3
• hdfs
• webhdfs

A lightweight stream processing library for Go

• stream-processing
• pipeline
• etl
• kafka

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

Real-time stream processing for python

• streaming-data
• python
• async
• real-time

Trill is a single-node query processor for temporal or streaming data.

• streaming-data
• temporal-data

HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.

• streaming
• database
• iot
• stream-processing

Machine is a workflow/pipeline library for processing data

• golang
• golang-library
• golang-package
• workflow

Window-Based Hybrid CPU/GPU Stream Processing Engine

• stream-processing
• hybrid
• multicore
• gpu

Stream Processing Made Easy

• machine-learning
• reactive-programming
• kafka
• reactive-systems

High performant ETL engine written in TypeScript

• data-processing
• etl
• etl-framework
• streaming-data