Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
- python
- luigi
- orchestration-framework
- scheduling
Repository Index
47,133 repositories indexed
1–19 of 19 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
Hazelcast is a unified real-time data platform combining stream processing with a fast data store, allowing customers to act instantly on data-in-motion for real-time insights.
- java
- hazelcast
- in-memory
- big-data
Apache KIE (Knowledge Is Everything) is the home of Drools, OptaPlanner, jBPM, and Kogito.
- rule-engine
- java
- dmn
- hacktoberfest
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
The fully compliant, embeddable high-performance Go MQTT v5 server for IoT, smarthome, and pubsub
- mqtt
- mqtt-broker
- mqtt-server
- mqtt-smarthome
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- data-engineering
- reverse-etl
- data-pipeline
- data-activation
Ncurses based XMPP client
- xmpp-client
- c
- xmpp
- chat
A lightweight, high-performance go mqtt server(v3.0|v3.1.1|v5.0) supporting distributed cluster
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
Home of Quamina, a fast pattern-matching library in Go
- event-driven
- go
- golang
- json
PipelinR is a lightweight command processing pipeline ❍ ⇢ ❍ ⇢ ❍ for your Java awesome app.
- java
- architecture
- service-layer
- pipes-and-filters
Universal BLE Smart Scale bridge. Captures body composition from Renpho, Xiaomi & 20+ others, syncs to Garmin Connect, Strava, MQTT (Home Assistant), InfluxDB, Webhooks, Ntfy & local files (CSV/JSONL). Headless CLI for Raspberry Pi, Linux, macOS & Windows.
🎀 Just the cutest and tiniest queue/deque implementation!
- stack
- queue
- npm-package
- data-structures
Distributed, Scalable, Fault-Tolerant, Minimalistic Workflow Engine - No DAGs, No YAML, No Cumbersome Diagrams, Just Code
- business-process
- kafka
- kafka-streams
- micorservices
Simple MQTT Client.
- mqtt
- client
- mqtt-client
- python
Production-grade Server-Sent Events (SSE) for Fiber v3. Event coalescing, priority lanes, topic wildcards, adaptive throttling, built-in auth, Prometheus metrics, graceful drain.
- event-streaming
- fasthttp
- fiber
- gofiber