Repository Index

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

• etl
• data-integration
• python
• postgresql

Python Stream Processing

• python
• stream-processing
• rust
• data-engineering

Crossbar.io - WAMP application router

• websocket
• wamp
• messaging
• mqtt

[Go] Lightweight eventbus with async compatibility for Go

• event-driven
• golang

Barebone MQTT broker that can run on any stream server, the node way

• mqtt
• mqtt-broker
• nodejs
• mqtt-server

:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop

The fully compliant, embeddable high-performance Go MQTT v5 server for IoT, smarthome, and pubsub

• mqtt
• mqtt-broker
• mqtt-server
• mqtt-smarthome

Apache Spark to Apache Cassandra connector

• cassandra
• scala
• spark

Messaging APIs for multi-platform

• messaging
• messenger
• line
• api-client

Python Stream Processing. A Faust fork

• asyncio
• python-streaming
• kafka
• redis

MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)

Redis Simple Message Queue

• rsmq
• queue
• message-queue
• fifo

Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.

• analytics
• bigquery
• data-modeling
• data-pipelines

Embulk: Pluggable Bulk Data Loader.

• embulk
• bulk-loader

Data processing & ETL framework for Ruby

• etl
• etl-ruby
• ruby
• data

Distributed Big Data Orchestration Service

• big-data
• bigdata
• orchestration
• configuration

EventMesh is a new generation serverless event middleware for building distributed event-driven applications.

• esb
• event-driven
• event-mesh
• event-governance

MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilise the resources of multiple clustered machines.

• mongooseim
• erlang
• xmpp
• jabber

Lightweight data stream processing engine for IoT edge

• edge
• sql
• iot
• edge-analytics

Lightweight and extensible compatibility layer between dataframe libraries!

• cudf
• ibis
• pandas
• polars

A distributed task scheduler for Dask

• pydata
• dask
• distributed-computing
• python

🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.

• data-engineering
• reverse-etl
• data-pipeline
• data-activation

Microservice native message and event store for Postgres

• event-store
• event-sourcing
• event-driven
• pub-sub

👜 Tiny and fast reactive/iterable programming library

• callbacks
• reactive
• iterables
• observables