Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
Repository Index
47,133 repositories indexed
1–5 of 5 · Most stars
BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.
- python3
- data-science
- machine-learning
- python
Read and write Neuroglancer datasets programmatically.
- neuroglancer
- volumetric-data
- python
- cloud
Blazingly fast data comparison tool for Python, powered by Rust. Compare massive CSV/Parquet datasets instantly.
GTFS Data Pipeline for TfNSW Bus Datasets
- gtfs
- gtfs-static
- gtfs-realtime
- tfnsw