SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.
- data-integration
- high-performance
- offline
- real-time
Repository Index
47,133 repositories indexed
1–20 of 20 · Most stars
Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
- bigquery
- copy-database
- data-ingestion
- data-integration
The open source and enterprise-ready webhooks service 🦀
- webhooks
- rust
- webhook-server
- webhook-service
StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.
- security
- kinesis
- serverless
- terraform
Concurrent and multi-stage data ingestion and data processing with Elixir
- elixir
- genstage
- data-ingestion
- data-processing
A framework for building messaging apps with .NET and C#.
- command-processor
- command-dispatcher
- brighter
- asynchronously
A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
- data
- ingestion
- replication
- management
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
- analytics
- bigquery
- data-modeling
- data-pipelines
The MongoDB Spark Connector
amqplib fork
- python
- amqp-client
- amqp
- amqp-connection
Streaming Postgres logical replication changes atleast-once over websockets
- postgresql
- logical-replication
- write-ahead-log
- websockets
Emissary is simple, yet 🗲FAST🗲 messaging library for decoupling messages (requests and events) and message handlers 🚀
- java
- jvm
- messaging
- requests
A tiny wrapper over amqp exchanges and queues 🚌 ✨
- rabbitmq
- eventbus
- microservices
- golang
For SwiftUI apps - a primitive component to make Chat-UI with stable prepending - no scroll jumps when loading older messages.
A micro workers class that easily manages a pool of threads to optimise simultaneous jobs and jobs endings, for MicroPython (used on Pycom modules & ESP32)
- pycom
- micropython
- lopy
- wipy
An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark.
- spark
- tweets
- twitter-data
- twitter-json
💬 Lightweight AlgoChat client for corvid-agent — GitHub Pages hosted
- algochat
- algorand
- blockchain
- encrypted-messaging
Data ingestion engine — Singer taps to Parquet in cloud buckets
Pipeline de ingestão e busca semântica de documentos para RAG — com rate limiting, circuit breaker e fila resiliente.