Scratch is a swiss army knife for big data.
- clickhouse
- data-warehouse
- hacktoberfest
- olap
108,094 visible repositories in the index
73–96 of 128 · Most stars
Scratch is a swiss army knife for big data.
omniparser: a native Golang ETL streaming parser and transform library for CSV, JSON, XML, EDI, text, etc.
ETL framework for .NET (Parser / Writer for CSV, Flat, Xml, JSON, Key-Value, Parquet, Yaml, Avro formatted files)
📈 Collect customer event data from your apps. (Note that this project only includes the API collector, not the visualization platform)
Polo travels through your database and creates sample snapshots so you can work with real world data in development.
Amazon Redshift Database Loader implemented in AWS Lambda
🌱 The ultimate solution for populating your MongoDB database.
Distributed Data Transfer Service for MySQL
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
An extensible Java framework for building event-driven applications that break up XML and non-XML data into chunks for data integration
Graph-based security analysis for everyone
The data transformation tool the open-source community deserves.
Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.
A decision-model-based ontology & knowledge-graph builder — builds governed knowledge graphs from your documents, every fact gated and traceable to its evidence.
Apache HBase Connectors
A fast to develop, fast to run, Go based toolkit for ETL and feature extraction on Hadoop.
📜 A CLI toolkit for extracting and working with your digital history
go-etl is a toolset for data extraction, transformation and loading.
A plugin to push to Algolia based on graphQl queries
Event-driven OneLake pipeline with automated Upsert merge and Python reporting
RDF-Centric Map/Reduce Framework and Freebase data conversion tool
Analyse GTFS Schedule data by importing it into a PostgreSQL database.
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.