Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
1–22 of 22 · Most stars
SubQuery is an Open, Flexible, Fast and Universal data indexing framework for web3. Our mission is to help developers create the decentralised products of the future.
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
A cluster computing framework for processing large-scale geospatial data
- cluster-computing
- spatial-sql
- geospatial
- spatial-analysis
Apache DataFusion Ballista Distributed Query Engine
- arrow
- big-data
- dataframe
- olap
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- data-engineering
- reverse-etl
- data-pipeline
- data-activation
Jupyter magics and kernels for working with remote Spark clusters
Trill is a single-node query processor for temporal or streaming data.
- streaming-data
- temporal-data
Data API Framework for AI Agents and Data Apps
A curated list of awesome Elixir and Command Query Responsibility Segregation (CQRS) resources.
- elixir
- cqrs
- event-sourcing
- domain-driven-design
Data-mining and Visualization components for RAD Studio
- visualization
- complex-structures
- database
- charting-library
A lightweight and easy-to-use CQRS + DDD library
- cqrs
- dotnet-standard
- command
- query
A set of open-source .NET libraries helping with DDD (domain-driven design) and CQRS (command query responsibility segregation).
- ddd
- cqrs
- nhibernate
- asp-net-core
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
- anomaly-detection
- bigquery
- cli
- data-engineering
ETL scripts for EOS.
- eos
- eosio
- etl
- apache-beam