Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Repository Index
47,133 repositories indexed
1–24 of 52 · Most stars
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
The most scalable and reliable MQTT broker for AI, IoT, IIoT and connected vehicles
Scalable real-time messaging server in a language-agnostic way. Self-hosted alternative to Pubnub, Pusher, Ably, socket.io, Phoenix.PubSub, SignalR. Set up once and forever.
Diskless Kafka® on S3. 10x Cost-Effective. No Cross-AZ Traffic Cost. Autoscale in seconds. Single-digit ms latency. Multi-AZ Availability.
- kafka
- diskless
- automq
- serverless
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
Zero-ETL, infinite possibilities. Live query APIs, code & more with SQL. No DB required.
- steampipe
- postgresql
- postgresql-fdw
- cloud
Migrate to PostgreSQL in a single command!
- postgresql
- migration
- mysql
- mssql
The live data layer for apps and AI agents. Create up-to-the-second views into your business, just using SQL
A lightweight message queue. Like AWS SQS and RSMQ but on Postgres.
The most widely used PHP client for RabbitMQ
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
Source-agnostic distributed change data capture system
Ultra-high performance reactive programming
Sync data from one Postgres database to another
World's most powerful open data catalog for building a high-performance, geo-distributed and federated metadata lake.
- datalake
- lakehouse
- metadata
- federated-query
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
- etl
- sql
- bigquery
- cloud-native
Database Subsetting and Relational Data Browsing Tool.
Self-hosted platform to unify wearable health data through one AI-ready API.
- wearables
- apple-health
- apple-watch
- fitness
High-performance time-series aggregation for PostgreSQL
- realtime
- stream-processing
- analytics
- sql
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
- etl
- data-integration
- python
- postgresql
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
- data-engineering
- reverse-etl
- data-pipeline
- data-activation