Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

• data
• python
• data-engineering
• data-lake

Open-source IoT Gateway - integrates devices connected to legacy and third-party systems with ThingsBoard IoT Platform using Modbus, CAN bus, BACnet, BLE, OPC-UA, MQTT, ODBC and REST protocols.

• iot
• iot-gateway
• gateway
• mqtt

:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark

• spark
• pyspark
• data-wrangling
• bigdata

Reverse ETL for the code-first data stack

• bigquery
• cli
• data-engineering
• dbt