Repository Index

The book's repo

A distributed Spark/Scala implementation of the isolation forest and extended isolation forest algorithms for unsupervised outlier detection, featuring support for scalable training and ONNX export for easy cross-platform inference.

• isolation-forest
• spark
• linkedin
• outlier-detection

The LinkedIn Fairness Toolkit (LiFT) is a Scala/Spark library that enables the measurement of fairness in large scale machine learning workflows.

• fairness
• fairness-ml
• fairness-ai
• linkedin

An Apache Spark framework for easy data processing, extraction as well as derivation for web archives and archival collections, developed at Internet Archive.

• archivespark
• spark-framework
• spark
• web-archiving

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

• spark
• hadoop
• webarchives
• analysis

PMML evaluator library for Apache Spark

A library that brings useful functions from various modern database management systems to Apache Spark

• spark
• postgres
• presto
• hive

Internet Archive's Sparkling Data Processing Library

An open-source toolkit for analyzing line-oriented JSON Twitter archives with Apache Spark.

• spark
• tweets
• twitter-data
• twitter-json

Gatling Extension for JDBC or Spark Thrift Server stress tests

• gatling
• jdbc
• thrift-server
• stress-testing

Spark plugin to retrieve metrics from a variety of cluster resources

• scala
• spark