Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
Repository Index
47,133 repositories indexed
1–17 of 17 · Most stars
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
Lightweight data stream processing engine for IoT edge
iWF is a Workflow-As-Code microservice orchestration platform offering an orchestration coding framework and service for building resilient, fault-tolerant, scalable long-running processes
- golang
- workflow
- distributed-system
- orchestration
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
- go
- data-integration
- etl
- data-pipeline
Simple but powerful DAG scheduler and dashboard
- directed-acyclic-graph
- airflow
- dashboard
- go
The type-safe, event-driven workflow orchestration library that scales with your business. Build robust, distributed workflows in Go with compile-time safety, automatic retries, and horizontal scaling out of the box. Integrate with your existing tech stack using adapters 🚀
- eventdrivenarchitecture
- golang
- kafka
- reflex
Lightweight stream processing engine for IoT
Machine is a workflow/pipeline library for processing data
- golang
- golang-library
- golang-package
- workflow
Time-series database and pub/sub messaging server for IoT and real-time devices, in Go: an embeddable storage engine with TTLs, wildcard topics and encryption, and a clustered server over gRPC, TCP and WebSocket with replication and rolling upgrades.
Distributed batch data processing framework
Lightweight SQL-based stream processing engine for IoT edge.
A lightweight Snowflake emulator built with Go and DuckDB for local development and testing
hypermatch is a high-performance ⚡ Go library designed for rapid matching of a large number of rules to events. It processes thousands of events per second 🚀 against extensive rule sets in-memory with minimal latency ⏱️.
- go
- golang
- matching
- matching-engine
A production-grade messaging abstraction for Go. It provides a unified API to decouple business logic from various brokers including Kafka, RabbitMQ, RocketMQ, NATS, Redis, and AWS SQS. Featuring built-in OpenTelemetry integration and engineered for high reliability.