201.robinhood / faust
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
- distributed-systems
- stream-processing
Ranked by Attention Score: stars gained over the last 30 complete days
201–300 of 344 by Attention Score · Oct 5, 2026 – Nov 3, 2026 (UTC)
Python Stream Processing
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
High-performance time-series aggregation for PostgreSQL
Compositional, streaming I/O library for Scala
JeroMQ is a pure Java implementation of the ZeroMQ messaging library, offering high-performance asynchronous messaging for distributed or concurrent applications.
Distributed Big Data Orchestration Service
Distributed Stream Processing
A platform that makes it easy for developers to build realtime, cost-effective, operations-focused applications
Dex : The Data Explorer -- A data visualization tool written in Java/Groovy/JavaFX capable of powerful ETL and publishing web visualizations.
Framework for building Event-Driven Microservices
A compatibility layer for event-driven abstractions
Workflow Automation Engine
PySpark test helper methods with beautiful error messages
C++11 thread-safe container for sharing data between threads (synchronized queue)
Embeddable and distributed in-memory alternative to Redis.
⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.
🚃 A library for handling mediator patterns and simplified CQRS patterns within an event-driven architecture, inspired by csharp MediatR library.
Confluent's Apache Kafka .NET client
Modular .NET messaging for commands, queries, events, durable inbox and outbox processing, sagas, and broker transports.
Data management for the sensor-edge-cloud continuum
GoEventBus — high-performance event bus with a decision layer.
Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency
A network event stream processing system, in Clojure.
Source-agnostic distributed change data capture system
The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️
A micro library providing Ruby objects with Publish-Subscribe capabilities
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
Kafka-style message streaming in Go. Built on NATS. Single binary, no JVM, no ZooKeeper.
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
Ruby and Rails efficient Kafka processing framework
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Python Stream Processing. A Faust fork
Microservice native message and event store for Postgres
Stream Processing and Complex Event Processing Engine
visual data prep powered by python
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
Data structure and algorithm library for go, designed to provide functions similar to C++ STL
Fast and reliable message broker built on top of Kafka.
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
Low-code tool for automating actions on real time data | Stream processing for the users.
pyspark methods to enhance developer productivity 📣 👯 🎉
A comprehensive Spark guide collated from multiple sources that can be referred to learn more about Spark or as an interview refresher.
This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
Tinode web chat using React
An HTTP bridge for Apache Kafka®
A powerful background job processing library for Clojure
TranslatedModern Reactive CQRS Architecture Microservice development framework based on DDD and EventSourcing
A Go port of the Java Streams API. Type-safe and functional Go Streams processing
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
Type-safe, persistent, embedded task queues and background job runner w/ SQLite. Web monitoring UI included.
Real-time decision features without streaming infra. Turn live events into product reflexes — no Kafka, no Flink, no feature store.
Formal specification and generation of verifiable binary parsers, message generators and protocol state machines
Time-series database and pub/sub messaging server for IoT and real-time devices, in Go: an embeddable storage engine with TTLs, wildcard topics and encryption, and a clustered server over gRPC, TCP and WebSocket with replication and rolling upgrades.
Reverse ETL for the code-first data stack
Execution runtime for intelligent agents with event-sourced, recoverable task orchestration.
A production-grade messaging abstraction for Go. It provides a unified API to decouple business logic from various brokers including Kafka, RabbitMQ, RocketMQ, NATS, Redis, and AWS SQS. Featuring built-in OpenTelemetry integration and engineered for high reliability.
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
🍴 Web applications made easy. Since 2011.
The most widely used PHP client for RabbitMQ
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Machine learning platform for Web developers
Project Flogo is an open source ecosystem of opinionated event-driven capabilities to simplify building efficient & modern serverless functions, microservices & edge apps.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Bunny is a popular, easy to use, mature Ruby client for RabbitMQ
Import CSV and JSON into PostgreSQL the easy way
A high available,high performance distributed messaging system.
Real-time stream processing for python
SQL-based streaming analytics platform at scale
Distributed Stream and Batch Processing
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
Chat SDK iOS - Open Source Mobile Messenger
Lua kafka client driver for the Openresty based on the cosocket API
Lightweight real-time big data streaming engine over Apache Pekko
A curated list of awesome Elixir and Command Query Responsibility Segregation (CQRS) resources.
A modern .NET framework for communication over RabbitMq
The MongoDB Spark Connector
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
Task management system based on .NET 8 with Microservices, DDD, CQRS, Event Sourcing and Testing Concepts
:surfer: Traceable, extendable and minimalist **event bus** implementation for Elixir with built-in **event store** and **event watcher** based on ETS.
RedisSMQ: A high-performance Redis message queue for Node.js
golang long polling library. Makes web pub-sub easy via HTTP long-poll servers and clients :smiley: :coffee: :computer:
iWF is a Workflow-As-Code microservice orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes
oneAPI Data Analytics Library (oneDAL)
An implementation of Actor, Publish-Subscribe, and CSP models in one rather small C++ framework. With performance, quality, and stability proved by years in the production.
.NET Stream Processing Library for Apache Kafka 🚀