Sign in

Repository Index

47,133 repositories indexed

Tags: Data pipelines and messaging
Tags

A repository shows when it has any selected Tag.

Clear Tags
Filters

Filters

Clear all filters

1–14 of 14 · Most stars

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

  • python
  • luigi
  • orchestration-framework
  • scheduling

Convert documents to structured data effortlessly. Unstructured is open-source ETL solution for transforming complex documents into clean, structured formats for language models. Visit our website to learn more about our enterprise grade Platform product for production grade workflows, partitioning, enrichments, chunking and embedding.

Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications

  • api
  • web-api
  • server-sent-events
  • subscriptions

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

MongooseIM is Erlang Solutions' robust, scalable and efficient XMPP server, aimed at large installations. Specifically designed for enterprise purposes, it is fault-tolerant and can utilise the resources of multiple clustered machines.

  • mongooseim
  • erlang
  • xmpp
  • jabber

Desbordante is a high-performance data profiler that is capable of discovering many different patterns in data using various algorithms. It also allows to run data cleaning scenarios using these algorithms. Desbordante has a console version and an easy-to-use web application.

Lightweight and Extensible Infrastructure for Building Web Applications - Web Application Framework

  • dotnet-core
  • efcore
  • aspnet-core
  • solid

Conflict-free merge for DataFrames, JSON, ML models & distributed agents — powered by CRDTs. The first merge library where every operation is mathematically guaranteed to converge.