For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
Repository Index
77,353 visible repositories in the index
1–24 of 110 · Most stars
Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
TranslatedAlibaba's MySQL binlog incremental subscription & consumption component
Apache Flink
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
SubQuery is an Open, Flexible, Fast and Universal data indexing framework for web3. Our mission is to help developers create the decentralised products of the future.
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
- rust
- database
- serverless
- bigdata
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
🧙 Build, run, and manage data pipelines for integrating and transforming data.
Apache Beam is a unified programming model for Batch and Streaming data processing.
- python
- java
- big-data
- beam
Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows
- fluentd
- c
- logging
- data-collector
Broadcast, Presence, and Postgres Changes via WebSockets
- elixir
- postgres
- postgresql
- realtime
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Flink CDC is a streaming data integration tool
- change-data-capture
- cdc
- batch
- data-integration
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
Translated🚗 Technical talks I have given personally... (topics: high-performance architecture, middleware principles and practice, cloud native, Golang, etc.)
- ppt
- python
- golang
- keynote
lakeFS - Data version control for your data lake | Git for data
- data-engineering
- data-versioning
- go
- object-storage
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
Apache Calcite
- geospatial
- calcite
- java
- big-data
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Distributed stream processing engine in Rust
- data
- dev-tools
- infrastructure
- kafka
a powerful mysql toolset with Go