Repository Index

A high-performance observability data pipeline.

• logs
• metrics
• observability
• forwarder

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

• kafka
• cpp
• kubernetes
• go

A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

• relational-database
• message-broker
• hadoop-ecosystem
• nosql

Python Stream Processing

• kafka-streams
• kafka
• python
• asyncio

Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.

• java
• hazelcast
• in-memory
• big-data

The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL

• rust
• database
• sql
• streaming

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

• asyncio
• kafka
• python
• rabbitmq

🦀 event stream processing for developers to collect and transform data in motion to power responsive data intensive applications.

• cloud-native
• streaming
• rust
• real-time

Distributed stream processing engine in Rust

• data
• dev-tools
• infrastructure
• kafka

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

• etl
• sql
• bigquery
• cloud-native

Kubernetes-native platform to run massively parallel data/streaming jobs

• kubernetes
• stream-processing
• data-processing
• k8s

Stream Processing and Complex Event Processing Engine

• streaming-data-processing
• cep
• stream-processing
• complex-event-processing

Framework for building Event-Driven Microservices

• microservices
• messaging
• event-driven-microservices
• stream-processing

HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.

• streaming
• database
• iot
• stream-processing

Microservices library - scalecube-services is a high throughput, low latency reactive microservices library built to scale. It features: API-Gateways, service-discovery, service-load-balancing, the architecture supports plug-and-play service communication modules and features. built to provide performance and low-latency real-time stream-processing

• microservices
• reactive-streams
• distributed-systems
• service-mesh

Machine is a workflow/pipeline library for processing data

• golang
• golang-library
• golang-package
• workflow

Scramjet Platform is a distributed runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters without exposing network ports to the public internet.

• nodejs
• stream-processing
• transform-as-a-service
• real-time

Stream Processing Made Easy

• machine-learning
• reactive-programming
• kafka
• reactive-systems

Stateful actors on Apache Flink 2.x and Java 21 — durable per-key state, exactly-once messaging, Kafka and Kinesis I/O, Kubernetes-native deployment. Continues the Apache Stateful Functions programming model on the modern Flink line.

• actor-model
• apache-flink
• distributed-systems
• event-driven