Repository Index

The simple, stupid batch framework for Java

• java
• batch-processing
• batch

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.

• go
• data-integration
• etl
• data-pipeline

Amazon Redshift Database Loader implemented in AWS Lambda

Distributed Data Transfer Service for MySQL

• mysql
• data-transfer
• data-migration
• replication

This repo maintains DM (a data migration platform) and TiCDC (change data capture for TiDB)

• cdc
• tidb
• mysql
• kafka

An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration

• java
• smooks
• enterprise-integration
• etl

The data platform for manufacturing

• oee
• iot

Powerful CSV & Excel Import experience for SaaS 🚀 Save months building data import experience from scratch 💰

• csv-import
• data-import
• data-mapping
• data-review

go-etl is a toolset for data extraction, transformation and loading.

• etl
• mysql
• go
• postgresql

A plugin to push to Algolia based on graphQl queries

• gatsby
• algolia
• search
• static-site-generator

Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.

This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.

• magento
• import
• products
• categories

Doc manager for Neo4j

• mongodb
• mongodb-support
• neo4j
• importer

🇪🇺 Warehouse-native Analytics compatible with Google Analytics and Matomo tracking protocols. Ingest into ClickHouse, BigQuery or CSV/Parquet on S3/GCS/local fs while maintaining complete control over your data.

• clickhouse
• ga4
• tracker
• bigquery

Enables search with Algolia in Hugo static sites

A simple data project for Steam data

• steam
• steam-data
• steam-api
• python

Java importer skeleton for complicated, business-logic-heavy high-performance Neo4j imports directly from SQL databases, CSV files, etc.

• neo4j
• java
• neo4j-graphaware-framework

Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.

• postgres
• postgresql
• python
• infectious-disease

Map of #ferguson walk out actions planned for Dec 1, 2014

ETL scripts for EOS.

• eos
• eosio
• etl
• apache-beam