Repository Index

Apache Spark - A unified analytics engine for large-scale data processing

• python
• scala
• r
• java

♞ lichess.org: the forever free, adless and open source chess server ♞

• scala
• chess
• play-framework
• non-profit

Scala 2 compiler and standard library. Scala 2 bugs at https://github.com/scala/bug; Scala 3 at https://github.com/scala/scala3

• scala
• scala-compiler
• scala-programming-language
• scala-library

A platform to build and run apps that are elastic, agile, and resilient. SDK, libraries, and hosted environments.

• reactive
• distributed-systems
• concurrency
• high-performance

Removes large or troublesome blobs like git-filter-branch does, but faster. And written in Scala

• git

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

• spark
• acid
• big-data
• analytics

The leader in Customer Data Infrastructure

• snowplow
• analytics
• data
• data-pipeline

Arnold Schwarzenegger based programming language

Apache OpenWhisk is an open source serverless cloud platform

• openwhisk
• apache
• serverless
• cloud

Lightweight, modular, and extensible library for functional programming.

Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark

• spark
• pyspark
• azure
• scala

State of the Art Natural Language Processing

• nlp
• natural-language-processing
• spark
• pyspark

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.

• scala
• sqs-interface
• elasticmq
• amazon-sqs

REST job server for Apache Spark

• spark
• rest-api
• spark-jobserver
• scala

Yet another JSON library for Scala

• json
• generic-derivation
• scala

Compositional, streaming I/O library for Scala

• scala
• stream
• streaming-computation
• fp

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

• spark
• hive
• sql
• thrift

Pure Scala Artifact Fetching

• scala
• sbt-plugin
• launcher
• scalajs

Apache Spark to Apache Cassandra connector

• cassandra
• scala
• spark

A data access library for Scala + Postgres.

• scala
• postgres
• functional-programming
• cats

Base classes to use when writing tests with Spark

MLeap: Deploy ML Pipelines to Production

• scikit-learn
• spark
• data-pipelines
• transformers

Code to accompany Advanced Analytics with Spark from O'Reilly Media