Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
47,133 repositories indexed
1–24 of 53 · Most stars
All-in-one IoT Platform - Device management, data collection, processing and visualization.
- visualization
- iot
- iot-platform
- mqtt
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
Open-Source Web UI for Apache Kafka Management
- kafka-ui
- kafka-brokers
- kafka
- kafka-streams
CMAK is a tool for managing Apache Kafka clusters
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
ZeroMQ core engine in C++, implements ZMTP/3.1
- zeromq
- zmq
- messaging
- concurrency
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Upserts, Deletes And Incremental Processing on Big Data.
- hudi
- apachehudi
- datalake
- bigdata
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
AI Infra / AI Orchestration / AI Control Plane
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
- workflow
- governance
- azkaban
- davinci
A 100% native C# implementation of ZeroMQ for .NET
- zeromq
- netmq
- messaging
- dotnet
:bus: Simple and lean service bus implementation for .NET
- rebus
- c-sharp
- service-bus
- messaging
JeroMQ is a pure Java implementation of the ZeroMQ messaging library, offering high-performance asynchronous messaging for distributed or concurrent applications.
- jeromq
- libzmq
- messaging
- networking
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
- data
- ingestion
- replication
- management
Distributed Big Data Orchestration Service
- big-data
- bigdata
- orchestration
- configuration
Clean APIs for data cleaning. Python implementation of R package Janitor
- pandas
- dataframe
- data
- cleaning-data
Gmqtt is a flexible, high-performance MQTT broker library that fully implements the MQTT protocol V3.x and V5 in golang
- mqtt
- mqtt-broker
- mqtt-protocol
- mqtt-server
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management