Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
- batch-processing
- kafka
- pathway
- python
Repository Index
47,133 repositories indexed
1–24 of 161 · Most stars
Cross-platform, customizable ML solutions for live and streaming media.
- mediapipe
- c-plus-plus
- computer-vision
- deep-learning
A realtime distributed messaging platform
High-Performance server for NATS.io, the cloud and edge native messaging system.
- go
- golang
- messaging
- message-bus
GoDS (Go Data Structures) - Sets, Lists, Stacks, Maps, Trees, Queues, and much more
- go
- golang
- data-structure
- map
IM Chat OpenClaw
- im
- chat
- messaging
- messenger
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
Instant messaging platform. Backend in Go. Clients: Swift iOS, Java Android, JS webapp, scriptable command line; chatbots
- chat
- chat-application
- chatapp
- go
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!
- kafka
- cpp
- kubernetes
- go
Sarama is a Go library for Apache Kafka.
- kafka
- kafka-client
- go
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
Building event-driven applications the easy way in Go.
- watermill
- event-driven
- event-sourcing
- sagas
Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
Golang client for NATS, the cloud native messaging system.
- go
- golang
- nats
- microservices
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
🪽 An open, easy, fast, reliable and battery-efficient solution for real-time communications
- api
- web-api
- server-sent-events
- subscriptions
a powerful mysql toolset with Go
Privacy and Security focused Segment-alternative, in Golang and React
- privacy
- warehouse-management
- data-warehouse
- customer-data
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
High performance, distributed and low latency publish-subscribe platform.
- pubsub
- networking
- emitter
- mqtt
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems