Repository Index

The AsyncAPI specification allows you to create machine-readable definitions of your asynchronous APIs.

• asyncapi
• reactive
• specification
• websockets

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

An ultra-lightweight and blazing-fast Messaging Bus/MQTT Broker for Edge & SDV

• mqtt
• mqtt-broker
• iot-edge
• nng

Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.

• data-science
• python
• dag
• data-engineering

This repository is a getting started guide to Singer.

• etl
• etl-framework
• python
• singer

zerocode-tdd is a community-developed, free, open-source, outcome-driven automated testing framework for Data Pipelines, ETL, REST API, Kafka(Data Streams), Databases and Load scenarios. all defined in simple JSON or YAML — with zero coding.

• soap
• assertions
• java
• api

BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.

• python3
• data-science
• machine-learning
• python

A refined core library for D programming language. The module has concurrency / collections / event / io / logging / text / serialization and more.

• event-driven
• asynchronous
• io
• concurrency

Declarative data quality engine. Define checks in YAML, run anywhere.

• airflow
• anomaly-detection
• data-contracts
• data-quality