Repository Index

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

• weak-supervision
• data-cleaning
• data-quality
• noisy-labels

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈

• ai-pipelines
• approximate-statistics
• statistical-properties
• data-quality

Generate the smallest test dataset that provably catches broken data-mapping rules in ETL pipelines.

Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.

• csv
• data-engineering
• data-quality
• diff

Validates GTFS Schedule datasets and exposes the results as metrics.