Repository Index

Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.

• database
• distributed-database
• distributed-sql-database
• sql

StreamAlert is a serverless, realtime data analysis framework which empowers you to ingest, analyze, and alert on data from any environment, using datasources and alerting logic you define.

• security
• kinesis
• serverless
• terraform

Compositional, streaming I/O library for Scala

• scala
• stream
• streaming-computation
• fp

Open-source ETL/ELT you deploy on your own servers or cloud. Built on DuckDB: no-code/low-code visual pipelines or SQL, 385 components, dbt, CDC, data quality, reverse ETL, lineage, MCP for AI agents. No vendor cloud, no per-row billing.

• data-engineering
• data-integration
• data-pipeline
• data-quality

Import CSV and JSON into PostgreSQL the easy way

Trill is a single-node query processor for temporal or streaming data.

• streaming-data
• temporal-data

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

• apache-spark
• big-data
• connected-components
• dataframe

🏎️ Streams & Reactive Programming paradigm for Go: declarative and composable API for event-driven applications

• event-driven
• generics
• go
• golang

:lipstick: Durable and asynchronous data imports for consuming data at scale and publishing testable SDKs.

• porter
• data-import
• framework
• data-transformation

Compositional pipelines

Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.

Powerful CSV & Excel Import experience for SaaS 🚀 Save months building data import experience from scratch 💰

• csv-import
• data-import
• data-mapping
• data-review

Distributed Temporal Graph Analytics with Apache Flink

• graph
• property-graph
• distributed-graph-analytics
• apache-flink

Universal BLE Smart Scale bridge. Captures body composition from Renpho, Xiaomi & 20+ others, syncs to Garmin Connect, Strava, MQTT (Home Assistant), InfluxDB, Webhooks, Ntfy & local files (CSV/JSONL). Headless CLI for Raspberry Pi, Linux, macOS & Windows.

• garmin
• typescript
• mqtt
• ble

Analyse GTFS Schedule data by importing it into a PostgreSQL database.

• gtfs
• postgresql
• postgres
• sql

This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.

• magento
• import
• products
• categories

Doc manager for Neo4j

• mongodb
• mongodb-support
• neo4j
• importer

Database synchronizer for Magento 2 (and Wordpress), based on Magerun2. Keeping development, staging and production in sync easily.

• magento2
• magento
• magerun2
• import

Open-source streaming SQL engine written in Rust using Apache Arrow and DataFusion. Supports continuous queries, temporal stream joins, tumbling/session windows, and CDC/Kafka connectors. Lightweight, embeddable, and sub-microsecond latency

Powerful pipes for TypeScript, that chain Promise and ADT for you 🚌 -> ⛰️ -> 🚠 -> 🏂 -> 🚀

• pipe
• pipeline
• kleisli
• ramda

Live import all your Google Sheets to your data warehouse

• data-warehouse
• data-vault
• etl
• google-sheets