Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
111,821 visible repositories in the index
1–24 of 33 · Most stars
Fancy stream processing made operationally mundane
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
Privacy and Security focused Segment-alternative, in Golang and React
RedisShake is a Valkey/Redis data processing and migration tool.
Sync MySQL data into elasticsearch
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
sq data wrangler
Flow collector, enricher and visualizer
Connect your company's data, let an agent draft an ontology over it, and load it into HelixDB as one graph you can explore and ask questions of. Hosted at foundry.helix-db.com, or run it on your own machine.
Bifrost ---- 面向生产环境的 MySQL,MariaDB,kafka 同步到Redis,MongoDB,ClickHouse,StarRocks,Doris,Kafka等服务的异构中间件
高性能表格数据导出器
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
MongoShake is a universal data replication platform based on MongoDB's oplog. Redundant replication and active-active replication are two most important functions. 基于mongodb oplog的集群复制工具,可以满足迁移和同步的需求,进一步实现灾备和多活功能。
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
Import CSV and JSON into PostgreSQL the easy way
a go daemon that syncs MongoDB to Elasticsearch in realtime. you know, for search.
go-stash is a high performance, free and open source server-side data processing pipeline that ingests data from Kafka, processes it, and then sends it to ElasticSearch.
Scratch is a swiss army knife for big data.
使用 golang 模仿的 Logstash。用于消费 Kafka 数据,处理后写入 ES、Clickhouse 等。
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
Distributed Data Transfer Service for MySQL