Repository Index

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.

• batch-processing
• kafka
• pathway
• python

Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.

• data
• pipeline
• data-analysis
• data-engineering

Self-hosted platform to unify wearable health data through one AI-ready API.

• wearables
• apple-health
• apple-watch
• fitness

Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.

• dataops
• dataops-platform
• elt
• open-source

Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.

• dataprep
• data-science
• datapreparation
• dataconnector

A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow

• etl
• data-integration
• python
• postgresql

MongoDB data stream pipeline tools by YouGov (adopted from MongoDB)

Python Streaming DataFrames for Kafka

• kafka
• python
• stream-processing
• data-engineering