Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
Nessie: Transactional Catalog for Data Lakes with Git-like semantics
- data
- spark
- java
- iceberg
A compatibility layer for event-driven abstractions
- stream-processing
- future
- clojure
- async
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.
- porter
- data-import
- framework
- data-transformation
Distributed Data Transfer Service for MySQL
- mysql
- data-transfer
- data-migration
- replication
Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.
- data-analytics
- data-cleaning
- data-cleansing
- data-engineering
Lightweight and Extensible Infrastructure for Building Web Applications - Web Application Framework
- dotnet-core
- efcore
- aspnet-core
- solid
OpenMessaging Specification
- queue
- transaction
- pull
- push
A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
go-etl is a toolset for data extraction, transformation and loading.
Machine is a workflow/pipeline library for processing data
- golang
- golang-library
- golang-package
- workflow
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.
A robust web archive analytics toolkit
- python
- web
- warc
- bigdata
Facilitates simple durable and asynchronous messaging in dotnet apps.
- async
- csharp
- dotnet
- messaging
Abstraction layer for simple rabbitMQ connection, messaging and administration
A production-grade messaging abstraction for Go. It provides a unified API to decouple business logic from various brokers including Kafka, RabbitMQ, RocketMQ, NATS, Redis, and AWS SQS. Featuring built-in OpenTelemetry integration and engineered for high reliability.