Blockchain Indexing Engine
Repository Index
72,862 visible repositories in the index
49–68 of 68 · Most stars
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
Event-driven OneLake pipeline with automated Upsert merge and Python reporting
- activator
- data-engineering
- data-factory
- data-pipeline
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
- antelope
- api
- big-data
- blockchain
Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.
This is a library that provides generic functionalities for the implementation of imports. In addition to maximum performance and optimized memory consumption, Pacemaker can also be used to implement imports in distributed scenarios that place the highest demands on speed and stability.
- magento
- import
- products
- categories
Bash script to export android content providers data to csv
Hadoop Data Integration with various databases, ftp servers, salesforce. Incremental update, dedup, append, merge your data on Hadoop.
Database synchronizer for Magento 2 (and Wordpress), based on Magerun2. Keeping development, staging and production in sync easily.
A blazing fast fully-automated CSV to database importer
- csv2sql
- csvs
- database
- csv-files
Go tool + library that publishes GIS vector data to GeoServer via PostGIS — walks a directory of shapefile / GeoJSON / GeoPackage / KML, loads each into PostGIS via GDAL/OGR, then registers the resulting tables as GeoServer feature types. Idempotent. Tested on GeoServer 2.27 LTS + 2.28 stable.
Code to download, process, and analyse the Online Encyclopedia of Integer Sequences
A Substrate indexing framework
Track Go benchmark performance over time by storing results in InfluxDB
Reverse ETL for the code-first data stack
- bigquery
- cli
- data-engineering
- dbt
EtlKit is a lightweight ETL (extract · transform · load) library and data integration toolbox for .NET. Build your own data pipelines programmatically in C# — readable, testable, and tracked in source control like the rest of your code.
- dotnet
- dotnet-core
- etl
- etl-framework
Live import all your Google Sheets to your data warehouse
- data-warehouse
- data-vault
- etl
- google-sheets
Real-time GTFS database loader and ORM library.
Map of #ferguson walk out actions planned for Dec 1, 2014
Instant data for apps on Bitcoin.
- clarity
- stacks
- typescript
- blockchain