Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Apache Beam is a unified programming model for Batch and Streaming data processing.

• python
• java
• big-data
• beam

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

• asyncio
• kafka
• python
• rabbitmq

Ultra-high performance reactive programming

StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.

• security
• kinesis
• serverless
• terraform

Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.

• flogo
• iot
• integration
• golang

Compositional, streaming I/O library for Scala

• scala
• stream
• streaming-computation
• fp

Ruby and Rails efficient Kafka processing framework

• karafka-framework
• kafka
• kafka-client
• kafka-clients

👜 Tiny and fast reactive/iterable programming library

• callbacks
• reactive
• iterables
• observables

Probabilistic data structures for processing continuous, unbounded streams.

• bloom-filter
• go
• stable-bloom-filters
• cuckoo-filter

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering

Distributed Stream Processing

• stream-processing
• stream-processor
• stream-processing-engine
• pony-language

🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications

• event-driven
• generics
• go
• golang

A stream processing framework to build stateless applications on Kafka

• kafka-streams
• clojure
• framework

Machine is a workflow/pipeline library for processing data

• golang
• golang-library
• golang-package
• workflow

Reactive queryable observable framework.

• reactive
• rx
• qbservable
• observable

Scramjet Platform is a distributed runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters without exposing network ports to the public internet.

• nodejs
• stream-processing
• transform-as-a-service
• real-time

A framework for machine-learning bots

Stream Processing Made Easy

• machine-learning
• reactive-programming
• kafka
• reactive-systems

Simple yet powerful live data computation framework.

• react
• javascript
• typescript
• event-stream

Python port of Scramjet framework

Simple yet powerful live data computation framework. C++ port of Scramjet framework.

• event-stream
• reactive-programming
• stream
• data-stream