A curated list of awesome big data frameworks, ressources and other awesomeness.
- awesome
- bigdata
- data-warehouse
- data-analytics
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
🌊 Online machine learning in Python
- incremental-learning
- machine-learning
- python
- online-learning
🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.
- cloud-native
- streaming
- rust
- real-time
Utils for streaming large files (S3, HDFS, gzip, bz2...)
- python
- s3
- hdfs
- webhdfs
A lightweight stream processing library for Go
Python Stream Processing
- python
- stream-processing
- rust
- data-engineering
Stream Processing and Complex Event Processing Engine
- streaming-data-processing
- cep
- stream-processing
- complex-event-processing
Python Streaming DataFrames for Kafka
Real-time stream processing for python
- streaming-data
- python
- async
- real-time
Trill is a single-node query processor for temporal or streaming data.
- streaming-data
- temporal-data
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
The database purpose-built for stream processing applications.
- stream-processing
- real-time
- sql
- interactive
Machine is a workflow/pipeline library for processing data
- golang
- golang-library
- golang-package
- workflow
Window-Based Hybrid CPU/GPU Stream Processing Engine
- stream-processing
- hybrid
- multicore
- gpu
Stream Processing Made Easy
- machine-learning
- reactive-programming
- kafka
- reactive-systems
High performant ETL engine written in TypeScript
- data-processing
- etl
- etl-framework
- streaming-data