A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
Repository Index
109,091 visible repositories in the index
73–94 of 94 · Most stars
📜 A CLI toolkit for extracting and working with your digital history
- chronicle
- chronicle-etl
- personal-data
- etl
Blockchain Indexing Engine
go-etl is a toolset for data extraction, transformation and loading.
- etl
- mysql
- go
- postgresql
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
RDF-Centric Map/Reduce Framework and Freebase data conversion tool
Analyse GTFS Schedule data by importing it into a PostgreSQL database.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
- antelope
- api
- big-data
- blockchain
Friendly utility to load your on-prem data, whether large or small, to Einstein Analytics Datasets, with useful features such as autoloading, dataflow control and dataset inspection.
A high performance data transfer tool that can be used to transfer data from Bitcoin Core blockchain files to a SQL Server database
This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.
- magento
- import
- products
- categories
Doc manager for Neo4j
🇪🇺 Warehouse-native Analytics compatible with Google Analytics and Matomo tracking protocols. Ingest into ClickHouse, BigQuery or CSV/Parquet on S3/GCS/local fs while maintaining complete control over your data.
- clickhouse
- ga4
- tracker
- bigquery
A Substrate indexing framework
Track Go benchmark performance over time by storing results in InfluxDB
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
- dotnet
- dotnet-core
- etl
- etl-framework
Open source industrial IoT connectivity gateway.
Data logistics system enabling real-time pathogen surveillance. Built for the Seattle Flu Study.
- postgres
- postgresql
- python
- infectious-disease
High performant ETL engine written in TypeScript
- data-processing
- etl
- etl-framework
- streaming-data
R COBOL DI (Data Integration) Package : Import COBOL CopyBook data files directly into R as properly structured data frames.
- r
- data-science
- dataframe
- data-frames
Real-time GTFS database loader and ORM library.
Analyse GTFS Schedule data by importing it into a DuckDB database.