Polyglot workflows without leaving the comfort of your technology stack.
- orchestration-framework
- scheduling
- workflow
- event-driven
Repository Index
47,133 repositories indexed
25–40 of 40 · Most stars
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
- go
- data-integration
- etl
- data-pipeline
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
- data-analytics
- data-cleaning
- data-cleansing
- data-engineering
Free and open source schema versioning and database migration made natively with .NET/6. NEW THIS MAY 2022! v1.3.15 released!
The data transformation tool the open-source community deserves.
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
Declarative data quality engine. Define checks in YAML, run anywhere.
- airflow
- anomaly-detection
- data-contracts
- data-quality
A detective for your data. Zero-config data quality monitoring — works with dbt, Postgres, BigQuery, Snowflake. No YAML.
- anomaly-detection
- bigquery
- cli
- data-engineering
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.
ETL scripts for EOS.
- eos
- eosio
- etl
- apache-beam
Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.
- collaborative-editing
- conflict-resolution
- crdt
- data-engineering
Open, audit-grade agentic data quality framework with portable industry packs
Data ingestion engine — Singer taps to Parquet in cloud buckets