ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
142,187 visible repositories in the index
25–48 of 101 · Most stars
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
The Data Transfer Project makes it easy for platforms to build interoperable user data portability features. We are establishing a common framework, including data models and protocols, to enable direct transfer of data both into and out of participating online service providers.
A Dockerized python Script to fetch Garmin health data and populate that in a InfluxDB Database, for visualization long term health trends with Grafana
LFX Community Data Platform (CDP)
Scalable and efficient data transformation framework - backwards compatible with dbt.
Django application and library for importing and exporting data with admin integration.
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Database Subsetting and Relational Data Browsing Tool.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
Spring Batch is a framework for writing batch applications using Java and Spring
Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
A high-performance Postgres replication engine written in Rust. Embed it in your Rust application or run it as a standalone binary.
🔮 Instill Core is a full-stack AI infrastructure tool for data, model and pipeline orchestration, designed to streamline every aspect of building versatile AI-first applications
Open-source low code data preparation library in python. Collect, clean and visualization your data in python with a few lines of code.
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
Python CLI utility and library for manipulating SQLite databases
Connect your company's data, let an agent draft an ontology over it, and load it into HelixDB as one graph you can explore and ask questions of. Hosted at foundry.helix-db.com, or run it on your own machine.
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Scalable data pre processing and curation toolkit for LLMs
MongoShake is a universal data replication platform based on MongoDB's oplog. Redundant replication and active-active replication are two most important functions. 基于mongodb oplog的集群复制工具,可以满足迁移和同步的需求,进一步实现灾备和多活功能。
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.