Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
1–24 of 33 · Most stars
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Sync data from one Postgres database to another
Database Subsetting and Relational Data Browsing Tool.
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
- database
- snapshot
- productivity
- activerecord
Amazon Redshift Database Loader implemented in AWS Lambda
🌱 The ultimate solution for populating your MongoDB database.
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Graph-based security analysis for everyone
Apache HBase Connectors
- database
- java
- hbase
A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
A plugin to push to Algolia based on graphQl queries
- gatsby
- algolia
- search
- static-site-generator
RDF-Centric Map/Reduce Framework and Freebase data conversion tool
Analyse GTFS Schedule data by importing it into a PostgreSQL database.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
- antelope
- api
- big-data
- blockchain
A high performance data transfer tool that can be used to transfer data from Bitcoin Core blockchain files to a SQL Server database
Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.
Database synchronizer for Magento 2 (and Wordpress), based on Magerun2. Keeping development, staging and production in sync easily.
A blazing fast fully-automated CSV to database importer
- csv2sql
- csvs
- database
- csv-files
Export your database from Lovable Cloud to Supabase
An import river similar to the elasticsearch mysql river