60,121 visible repositories in the index
25–48 of 54 · Most stars
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 426 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
visual data prep powered by python
Dex : The Data Explorer -- A data visualization tool written in Java/Groovy/JavaFX capable of powerful ETL and publishing web visualizations.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
Distributed Data Transfer Service for MySQL
Import, validate, query, and export GTFS Schedule and Realtime data with SQLite, PostgreSQL, and MySQL.
This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
The data transformation tool the open-source community deserves.
High performance Kafka consumer for InfluxDB. Supports collectd message formats.
go-etl is a toolset for data extraction, transformation and loading.
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
Event-driven OneLake pipeline with automated Upsert merge and Python reporting
Analyse GTFS Schedule data by importing it into a PostgreSQL database.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.
A high performance data transfer tool that can be used to transfer data from Bitcoin Core blockchain files to a SQL Server database
This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.
A blazing fast fully-automated CSV to database importer
Insert incoming MQTT values into InfluxDB. Follows mqtt-smarthome architecture.
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
High performant ETL engine written in TypeScript