Always know what to expect from your data.
- pipeline-tests
- dataquality
- datacleaning
- datacleaner
Repository Index
47,133 repositories indexed
1–24 of 28 · Most stars
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Evidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics.
- data-drift
- jupyter-notebook
- pandas-dataframe
- machine-learning
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
A light-weight, flexible, and expressive statistical data testing library
- pandas
- validation
- schema
- dataframes
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.
- machine-learning
- ml
- model-validation
- data-validation
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
Database Subsetting and Relational Data Browsing Tool.
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
- data-science
- python
- dag
- data-engineering
The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.
- clickhouse
- kakfa
- sql
- redpanda
A complete, fully tested and documented data structure library written in pure JavaScript.
- javascript
- data-structures
- collections
- collection
A curated list of awesome system integration software and resources.
- esb
- bpm
- ipaas
- apimanagement
Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)
Awesome CMS Core is an open source CMS built using ASP.Net Core & ReactJS with module seperation concern in mind and provide lastest trend of technology like .Net Core, React, Webpack, SASS, Background Job, Message Queue.
- cms
- dotnetcore
- dotnet-standard
- grunt
Lightweight and Extensible Infrastructure for Building Web Applications - Web Application Framework
- dotnet-core
- efcore
- aspnet-core
- solid
RabbitMQ wire tap and swiss army knife
- rabbitmq
- tap
- wiretap
- golang
k6 extension to load test Apache Kafka with support for various serialization formats, SASL, TLS, compression, Schema Registry client and beyond
BENERATOR is a leading software solution to generate, obfuscate, pseudonymize and migrate data for development, testing, and training purposes with a model-driven approach.
- testdata
- anonymization
- migration
- performance-testing
🎀 Just the cutest and tiniest queue/deque implementation!
- stack
- queue
- npm-package
- data-structures
Big Data essentials: Hadoop, MapReduce, Spark. Explore tutorials and demos in Jupyter notebooks—most are self-contained and live, ready to run with a click.
Realistic synthetic events for testing, demos, and pipelines — streamed live or generated in bulk
- clickhouse
- data-engineering
- data-generation
- devops
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
- dotnet
- dotnet-core
- etl
- etl-framework
A JMeter plug-in that enables you to send test results to a Kafka server