Fancy stream processing made operationally mundane
- message-queue
- stream-processing
- streaming-data
- message-bus
98,836 visible repositories in the index
1–12 of 12 · Most stars
Fancy stream processing made operationally mundane
Data pipelines for cloud config and security data. Build cloud asset inventory, CSPM, FinOps, and vulnerability management solutions. Extract from AWS, Azure, GCP, and 70+ cloud and SaaS sources.
RedisShake is a Valkey/Redis data processing and migration tool.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
sq data wrangler
高性能表格数据导出器
Build data pipelines with SQL and Python, ingest data from different sources, add quality checks, and build end-to-end flows.
OLake Go: open-source replication from databases, Kafka and S3 to Apache Iceberg and Parquet. Sources: Postgres, MySQL, MongoDB, Oracle, MSSQL, DB2, Kafka, S3. Formerly datazip-inc/olake. Table maintenance lives in datazip-inc/olake-fusion.
Import CSV and JSON into PostgreSQL the easy way
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
go-etl is a toolset for data extraction, transformation and loading.
Go tool + library that publishes GIS vector data to GeoServer via PostGIS — walks a directory of shapefile / GeoJSON / GeoPackage / KML, loads each into PostGIS via GDAL/OGR, then registers the resulting tables as GeoServer feature types. Idempotent. Tested on GeoServer 2.27 LTS + 2.28 stable.