Repository Index

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

• gbdt
• gbrt
• gbm
• distributed-systems

Apache Flink

• scala
• java
• big-data
• flink

Translatedflink learning blog. http://www.54tianzhisheng.cn/ Covers Flink introduction, concepts, principles, hands-on practice, performance tuning, source code analysis, and more. Includes learning examples involving Flink Connector, Metrics, Library, DataStream API, Table API & SQL, and also shares large-scale project cases of Flink in production (PV/UV, log storage, real-time deduplication of 10 billion records, monitoring and alerting). You are welcome to support my column 《大数据实时计算引擎 Flink 实战与性能优化》 ("Big Data Real-Time Computing Engine Flink: Hands-On Practice and Performance Optimization")

• flink
• kafka
• elasticsearch
• spark

TranslatedFocused on big data learning and interviews, the road to becoming a big data god begins. Flink/Spark/Hadoop/Hbase/Hive...

• flink
• spark
• hadoop
• hdfs

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

• spark
• acid
• big-data
• analytics

Upserts, Deletes And Incremental Processing on Big Data.

• hudi
• apachehudi
• datalake
• bigdata

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.

• workflow
• governance
• azkaban
• davinci

Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

• clickhouse
• kakfa
• sql
• redpanda

Low-code tool for automating actions on real time data | Stream processing for the users.

• flink
• flink-kafka
• apache-flink
• gui

Benchmarks for Low Latency (Streaming) solutions including Apache Storm, Apache Spark, Apache Flink, ...

• benchmarks
• low-latency
• streaming

Distributed Temporal Graph Analytics with Apache Flink

• graph
• property-graph
• distributed-graph-analytics
• apache-flink

The Open Source Controller

• iot
• homeautomation
• esp8266
• arduino

Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.

• analytics
• feature-store
• fraud-detection
• python

Stateful actors on Apache Flink 2.x and Java 21 — durable per-key state, exactly-once messaging, Kafka and Kinesis I/O, Kubernetes-native deployment. Continues the Apache Stateful Functions programming model on the modern Flink line.

• actor-model
• apache-flink
• distributed-systems
• event-driven