For developers, who are building real-time data-driven applications, Redis is the preferred, fastest, and most feature-rich cache, data structure server, and document and vector query engine.
Repository Index
72,862 visible repositories in the index
1–24 of 103 · Most stars
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
- change-data-capture
- kafka-connect
- apache-kafka
- debezium
Node-based Visual Programming Toolbox
- photogrammetry
- computer-vision
- 3d-reconstruction
- alicevision
Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Cleanlab's open-source library is the standard data-centric AI package for data quality and machine learning with messy, real-world data and labels.
- weak-supervision
- data-cleaning
- data-quality
- noisy-labels
Data Agent Ready Warehouse : One for Analytics, Search, AI, Python Sandbox. — rebuilt from scratch. Unified architecture on your S3.
- rust
- database
- serverless
- bigdata
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
- data
- data-cleaning
- data-pipelines
- data-processing
Broadcast, Presence, and Postgres Changes via WebSockets
- elixir
- postgres
- postgresql
- realtime
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
TranslatedDataX integrated with a visual page: select a data source and generate a data synchronization task with one click. Supports data sources such as RDBMS, Hive, HBase, ClickHouse, and MongoDB, and batch creation of RDBMS data synchronization tasks. Integrates an open-source scheduling system, and supports distributed and incremental data synchronization, real-time viewing of running logs, monitoring of executor resources, KILL of running processes, encryption of data source information, and more.
Apache Iggy: Hyper-Efficient Message Streaming at Laser Speed
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
TranslatedDeepAnalyze is the first agentic LLM for autonomous data science. 🎈Your AI data analyst, automatically analyzes large amounts of data and generates professional analysis reports with one click!
SQL Translator is a tool for converting natural language queries into SQL code using artificial intelligence. This project is 100% free and open source.
- data-analysis
- data-engineering
- dataquery
- datascience
Cartography is a Python tool that pulls infrastructure assets and their relationships into a Neo4j graph database.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
A distributed MQTT message broker based on Erlang/OTP. Built for high quality & Industrial use cases. The VerneMQ mission is active & the project maintained. Thank you for your support!
- mqtt
- m2m
- scalable
- broker
A Dockerized python Script to fetch Garmin health data and populate that in a InfluxDB Database, for visualization long term health trends with Grafana
- garmin
- garmin-api
- garmin-connect
- garmin-data