Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
Home of Quamina, a fast pattern-matching library in Go
Data validation toolkit for assessing and monitoring data quality.
Compositional pipelines
PipelinR is a lightweight command processing pipeline ❍ ⇢ ❍ ⇢ ❍ for your Java awesome app.
- java
- architecture
- service-layer
- pipes-and-filters
Messaging patterns for Backbone applications.
Simple but powerful DAG scheduler and dashboard
- directed-acyclic-graph
- airflow
- dashboard
- go
Streaming reactive and dataflow graphs in Python
This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)
A simple pubsub package for go.
Haskell on Apache Spark.
MySQL to PostgreSQL replica system
Canonical GTFS Validator project for schedule (static) files.
- gtfs-dataset
- gtfs-validator
- gtfs-zip
- gtfs-feed
Series and Panels for Real-time and Exploratory Analysis of Data Streams
Web tool for Avro Schema Registry |
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
- java
- smooks
- enterprise-integration
- etl
A stream processing framework to build stateless applications on Kafka
- kafka-streams
- clojure
- framework
The data platform for manufacturing
An open source, a high-performance hybrid integration platform that allows developers quick integration with any application, data, or system.
Automatic data change tracking for PostgreSQL
Enhanced PostgreSQL logical replication
Phoebus is a distributed framework for large scale graph processing written in Erlang.
⏪️ Go package providing multiple queue implementations. Developed in a thread-safe generic way.
- generic
- golang
- queue
- thread-safe
Postgres change events (CDC) in Elixir [WAL]