Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
1–23 of 23 · Most stars
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Sync data from one Postgres database to another
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
Database Subsetting and Relational Data Browsing Tool.
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- data-engineering
- reverse-etl
- data-pipeline
- data-activation
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
- data-engineering
- data-integration
- data-pipeline
- data-quality
Import CSV and JSON into PostgreSQL the easy way
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
Universal BLE Smart Scale bridge. Captures body composition from Renpho, Xiaomi & 20+ others, syncs to Garmin Connect, Strava, MQTT (Home Assistant), InfluxDB, Webhooks, Ntfy & local files (CSV/JSONL). Headless CLI for Raspberry Pi, Linux, macOS & Windows.
Blockchain Indexing Engine
go-etl is a toolset for data extraction, transformation and loading.
Analyse GTFS Schedule data by importing it into a PostgreSQL database.
A blazing fast fully-automated CSV to database importer
- csv2sql
- csvs
- database
- csv-files
Go tool + library that publishes GIS vector data to GeoServer via PostGIS — walks a directory of shapefile / GeoJSON / GeoPackage / KML, loads each into PostGIS via GDAL/OGR, then registers the resulting tables as GeoServer feature types. Idempotent. Tested on GeoServer 2.27 LTS + 2.28 stable.
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.
- postgres
- postgresql
- python
- infectious-disease
Instant data for apps on Bitcoin.
- clarity
- stacks
- typescript
- blockchain