Sign in

Repository Index

47,133 repositories indexed

Tag: Data pipelines and messaging
Tag
Filters

Filters

Clear all filters

1–24 of 135 · Most stars

Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.

  • python
  • luigi
  • orchestration-framework
  • scheduling

SubQuery is an Open, Flexible, Fast and Universal data indexing framework for web3. Our mission is to help developers create the decentralised products of the future.

The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles

Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.

Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.

Generic command line non-JVM Apache Kafka producer and consumer

🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications

  • api
  • web-api
  • server-sent-events
  • subscriptions

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!

  • mqtt
  • m2m
  • scalable
  • broker

The open source and enterprise-ready webhooks service 🦀

  • webhooks
  • rust
  • webhook-server
  • webhook-service

A micro library providing Ruby objects with Publish-Subscribe capabilities

  • ruby
  • events
  • publish-subscribe
  • pubsub

DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.