60,121 visible repositories in the index
1–16 of 16 · Most stars
Open Lakehouse Format for Multimodal AI. Convert from Parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, and PyTorch with more integrations coming..
A lightweight data processing framework built on DuckDB and 3FS.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Dinky is a real-time data development platform based on Apache Flink, enabling agile data development, deployment and operation.
sq data wrangler
Lightweight and extensible compatibility layer between dataframe libraries!
Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 426 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.
High performance Rust stream processing engine seamlessly integrates AI capabilities, providing powerful real-time data processing and intelligent analysis.
Cross-platform beanstalkd queue server admin console.
The data transformation tool the open-source community deserves.
Reverse ETL for the code-first data stack
Declarative data quality engine. Define checks in YAML, run anywhere.
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
Analyse GTFS Schedule data by importing it into a DuckDB database.
Open, audit-grade agentic data quality framework with portable industry packs
Data ingestion engine — Singer taps to Parquet in cloud buckets