For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
Repository Index
63,269 visible repositories in the index
1–24 of 123 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
SeaweedFS is a distributed storage system for object storage (S3), file systems, Iceberg table buckets, and Lance vector buckets, designed to handle billions of files with O(1) disk access and effortless horizontal scaling.
Apache Kafka - A distributed event streaming platform
A realtime distributed messaging platform
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
High-Performance server for NATS.io, the cloud and edge native messaging system.
- go
- golang
- messaging
- message-bus
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles
The Open Context Layer for Data and AI , OpenMetadata is the open platform for building trusted data context and business semantics for humans, AI assistants, and agents.
- metadata
- datadiscovery
- dataquality
- data-profiling
Apache Pulsar - distributed pub-sub messaging system
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
Type-safe, distributed orchestration of agents, ML pipelines, and real-time inference on your k8s — in pure Python with async/await, also other languages (rust, go and ts)
- machine-learning
- scale
- workflow
- mlops
Distributed Application Framework for .NET
- masstransit
- rabbitmq
- c-sharp
- azure-service-bus
Enterprise job scheduling middleware with distributed computing ability.
- scheduler
- workflow
- distributed
- mapreduce
Broadcast, Presence, and Postgres Changes via WebSockets
- elixir
- postgres
- postgresql
- realtime
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio