Repository Index

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

Scalable data pre processing and curation toolkit for LLMs

• data-curation
• llm
• data
• data-prep

Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match

Generate the smallest test dataset that provably catches broken data-mapping rules in ETL pipelines.

Field-level assurance for data pipelines: mapping-driven source-to-target validation, fault-injection test scoring and explainable risk-adaptive test prioritization.

Declarative data quality engine. Define checks in YAML, run anywhere.

• airflow
• anomaly-detection
• data-contracts
• data-quality

Validates tabular CSV data using predefined validations, inspired from its Python homologue "Great Expectations".

• common-lisp
• csv
• data-validation
• great-expectations