Incremental engine for long horizon agents 🌟 Star if you like it!
- ai
- change-data-capture
- data-indexing
- etl
Repository Index
47,133 repositories indexed
1–18 of 18 · Most stars
pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, Neptune, OpenSearch, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).
:elephant: Elasticsearch real-time search and analytics natively integrated with Hadoop
Python Streaming DataFrames for Kafka
Download, unpack from a ZIP/TAR/GZ/BZ2 archive, parse, correct, convert units and import Google Spreadsheets, XLS, ODS, XML, CSV, HTML, etc. into your ActiveRecord models. Uses RemoteTable gem internally.
📜 A CLI toolkit for extracting and working with your digital history
- chronicle
- chronicle-etl
- personal-data
- etl
Blockchain Indexing Engine
A plugin to push to Algolia based on graphQl queries
- gatsby
- algolia
- search
- static-site-generator
An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.
- archivespark
- spark-framework
- spark
- web-archiving
Scalable Full History API Solution for Antelope (former EOSIO) based blockchains
- antelope
- api
- big-data
- blockchain
Please note that the warc-indexer tool & code is now supported by NetArchiveSuite. The 'warc-indexer' directory and code that exists in this repo is now only for reference. For support and issues of 'warc-indexer', please communicate with NetArchiveSuite.
Enables search with Algolia in Hugo static sites
An import river similar to the elasticsearch mysql river
Insert incoming MQTT values into InfluxDB. Follows mqtt-smarthome architecture.
Internet Archive's Sparkling Data Processing Library
Jmeter JTL parsing with Logstash for Elasticseacrh and Influxdb ...
- jmeter
- logstash
- filebeat
- elasticsearch
Send MQTT messages to Elasticsearch
Reads EVE files into SQL as well as search stored data.