Sign in

Repository Index

61,275 visible repositories in the index

Tags: Data pipelines and messaging
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–23 of 23 · Most stars

Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

Apache Camel is an open source integration framework with 350+ connectors. Write routes in Java, YAML, or XML. Run on Spring Boot, Quarkus, or standalone. Apache License 2.0.

Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days

The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

Lightweight and Extensible Infrastructure for Building Web Applications - Web Application Framework

  • dotnet-core
  • efcore
  • aspnet-core
  • solid

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

🚃 A library for handling mediator patterns and simplified CQRS patterns within an event-driven architecture, inspired by csharp MediatR library.

  • cqrs
  • golang
  • mediator
  • mediator-pattern

Modular .NET messaging for commands, queries, events, durable inbox and outbox processing, sagas, and broker transports.

  • cqrs
  • cqs
  • csharp
  • dotnet-core

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

  • archivespark
  • spark-framework
  • spark
  • web-archiving

Scramjet Platform is a distributed runtime supervisor that can run data processing programs called Sequences and manage local resources on any Linux server, Docker on small edge servers, and even large-scale Kubernetes clusters in the cloud or datacenters without exposing network ports to the public internet.

Production-grade Server-Sent Events (SSE) for Fiber v3. Event coalescing, priority lanes, topic wildcards, adaptive throttling, built-in auth, Prometheus metrics, graceful drain.