Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
72,862 visible repositories in the index
1–24 of 48 · Most stars
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
TranslatedDataX integrated with a visual page: select a data source and generate a data synchronization task with one click. Supports data sources such as RDBMS, Hive, HBase, ClickHouse, and MongoDB, and batch creation of RDBMS data synchronization tasks. Integrates an open-source scheduling system, and supports distributed and incremental data synchronization, real-time viewing of running logs, monitoring of executor resources, KILL of running processes, encryption of data source information, and more.
Cartography is a Python tool that pulls infrastructure assets and their relationships into a Neo4j graph database.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
A Dockerized python Script to fetch Garmin health data and populate that in a InfluxDB Database, for visualization long term health trends with Grafana
- garmin
- garmin-api
- garmin-connect
- garmin-data
Sync data from one Postgres database to another
Download and parse data from Garmin Connect or a Garmin watch, FitBit CSV, and MS Health CSV files into and analyze data in Sqlite serverless databases with Jupyter notebooks.
- garmin
- python
- health
- database
Database Subsetting and Relational Data Browsing Tool.
The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure
- claude-code
- codex
- data-processing
- harness-engineering
sq data wrangler
深圳地铁大数据客流分析系统🚇🚄🌟
A high-performance Postgres replication engine written in Rust. Embed it in your Rust application or run it as a standalone binary.
- logical-replication
- postgres
- replication
- rust
Python CLI utility and library for manipulating SQLite databases
- sqlite
- python
- datasette
- sqlite-database
Connect your company's data, let an agent draft an ontology over it, and load it into HelixDB as one graph you can explore and ask questions of. Hosted at foundry.helix-db.com, or run it on your own machine.
- ai-agents
- data-integration
- golang
- graph-database
Copy a Postgres database to a target Postgres server (pg_dump | pg_restore on steroids)
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
- database
- snapshot
- productivity
- activerecord
Amazon Redshift Database Loader implemented in AWS Lambda
🌱 The ultimate solution for populating your MongoDB database.
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
Graph-based security analysis for everyone
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
- graphrag
- information-extraction
- knowledge-graph
- llm