Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 427 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks.
Hop Orchestration Platform
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Apache InLong - a one-stop, full-scenario integration framework for massive data
Actively maintained successor to Alibaba DataX — a fast, versatile, open-source ETL tool for 30+ RDBMS and NoSQL data sources.
visual data prep powered by python
- data
- data-pipelines
- etl
- structured-data
TypeScript ETL toolkit for indexing Ethereum, Solana, and Substrate data, sourced from SQD Network.
a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
Python Extract Transform and Load Tables of Data
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
Alpakka is a Reactive Enterprise Integration library for Java and Scala, based on Reactive Streams and Akka.
- integration
- reactive
- akka
- akka-stream
Python Fast Dataflow programming framework for Data pipeline work( Web Crawler,Machine Learning,Quantitative Trading.etc)
Kylo is a data lake management software platform and framework for enabling scalable enterprise-class data lakes on big data technologies such as Teradata, Apache Spark and/or Hadoop. Kylo is licensed under Apache 2.0. Contributed by Teradata Inc.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
Healthcare data infrastructure for next generation care delivery.
- api
- healthcare
- open-source
- soc2
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
- porter
- data-import
- framework
- data-transformation
Distributed Data Transfer Service for MySQL
- mysql
- data-transfer
- data-migration
- replication
Import, validate, query, and export GTFS Schedule and Realtime data with SQLite, PostgreSQL, and MySQL.
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
- java
- smooks
- enterprise-integration
- etl
The data transformation tool the open-source community deserves.
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
- graphrag
- information-extraction
- knowledge-graph
- llm