Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
A lightweight data processing framework built on DuckDB and 3FS.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
DuckLake is an integrated data lake and catalog format
Lightweight and extensible compatibility layer between dataframe libraries!
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 427 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
dbt adapter for DuckDB
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
- flow
- rust
- rust-lang
- stream
ClickBench: a Benchmark For Analytical Databases
The data transformation tool the open-source community deserves.
Reverse ETL for the code-first data stack
Declarative data quality engine. Define checks in YAML, run anywhere.
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
Analyse GTFS Schedule data by importing it into a DuckDB database.
Open, audit-grade agentic data quality framework with portable industry packs
Data ingestion engine — Singer taps to Parquet in cloud buckets