Trending repositories

1.apache / spark

Apache Spark - A unified analytics engine for large-scale data processing

• python
• scala
• r
• java
• big-data
• jdbc
• +2 more

2.delta-io / delta

An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs

• spark
• acid
• big-data
• analytics
• delta-lake

3.scala / scala

Scala 2 compiler and standard library. Scala 2 bugs at https://github.com/scala/bug; Scala 3 at https://github.com/scala/scala3

• scala
• scala-compiler
• scala-programming-language
• scala-library
• jvm-languages
• functional-programming
• +1 more

4.softwaremill / elasticmq

In-memory message queue with an Amazon SQS-compatible interface. Runs stand-alone or embedded.

• scala
• sqs-interface
• elasticmq
• amazon-sqs
• aws
• aws-sqs
• +1 more

5.apache / pekko

Build highly concurrent, distributed, and resilient message-driven applications using Java/Scala

• actor-model
• cloud-native
• concurrency
• distributed-actors
• distributed-systems
• high-performance
• +4 more

6.gatling / gatling

Modern Load Testing as Code

• netty
• scala
• loadtesting
• automation
• gatling
• load-testing
• +13 more

7.gitbucket / gitbucket

A Git platform powered by Scala with easy installation, high extensibility & GitHub API compatibility

• scala
• gitbucket
• git
• scalatra

8.apache / kyuubi

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.

• spark
• hive
• sql
• thrift
• jdbc
• spark-sql
• +4 more

9.snowplow / snowplow

The leader in Customer Data Infrastructure

• snowplow
• analytics
• data
• data-pipeline
• data-collection
• product-analytics
• +3 more

10.awslabs / deequ

Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.

• dataquality
• spark
• unit-testing
• scala

11.apache / openwhisk

Apache OpenWhisk is an open source serverless cloud platform

• openwhisk
• apache
• serverless
• cloud
• faas
• functions-as-a-service
• +5 more

12.playframework / playframework

The Community Maintained High Velocity Web Framework For Java and Scala.

• scala
• java
• reactive
• web-framework
• restful
• play
• +5 more

13.lhartikk / ArnoldC

Arnold Schwarzenegger based programming language

14.JohnSnowLabs / spark-nlp

State of the Art Natural Language Processing

• nlp
• natural-language-processing
• spark
• pyspark
• named-entity-recognition
• sentiment-analysis
• +14 more

15.Netflix / atlas

In-memory dimensional time series database.

16.graphframes / graphframes

GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs

• apache-spark
• big-data
• connected-components
• dataframe
• dataframes
• graphs
• +8 more

17.yahoo / CMAK

CMAK is a tool for managing Apache Kafka clusters

• kafka
• scala
• cluster-management
• big-data

18.apache / predictionioArchived

PredictionIO, a machine learning server for developers and ML engineers.

• scala
• big-data
• predictionio

19.prisma / prisma1Archived

💾 Database Tools incl. ORM, Migrations and Admin UI (Postgres, MySQL & MongoDB) [deprecated]

• orm
• database
• graphql
• datamapper
• serverless
• dao
• +1 more

20.coursier / coursier

Pure Scala Artifact Fetching

• scala
• sbt-plugin
• launcher
• scalajs
• scala-native

21.TouK / nussknacker

Low-code tool for automating actions on real time data | Stream processing for the users.

• flink
• flink-kafka
• apache-flink
• gui
• touk
• real-time
• +13 more

22.apache / livy

Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.

• livy
• bigdata
• spark
• apachelivy

23.bigdatagenomics / adam

ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.

• spark
• big-data
• bioinformatics
• genomics
• parquet
• avro
• +4 more

24.combust / mleap

MLeap: Deploy ML Pipelines to Production

• scikit-learn
• spark
• data-pipelines
• transformers
• tensorflow
• scala
• +1 more

25.scalalandio / chimney

Scala library for boilerplate-free, type-safe data transformations

• scala
• boilerplate
• generic-programming
• automatic-conversion
• auto-mapping
• data-rewriting
• +12 more

26.circe / circe

Yet another JSON library for Scala

• json
• generic-derivation
• scala

27.apache / cassandra-spark-connector

Apache Spark to Apache Cassandra connector

• cassandra
• scala
• spark

28.softwaremill / sttp

The Scala HTTP client you always wanted!

• scala
• http
• httpclient
• interpolator
• uri
• asynchronous
• +12 more

29.filodb / FiloDB

Distributed Prometheus time series database

• scala
• time-series
• database
• distributed
• prometheus
• metrics
• +2 more

30.scalacenter / bloop

Bloop is a build server and CLI tool to compile, test and run Scala fast from any editor or build tool.

• scala
• productivity
• developer-tools
• compilation-server
• java
• build-system
• +1 more

31.galax-io / gatling-picatinny

Gatling DSL extensions — feeders, profile management, transactions, assertions, Redis integration, and utility functions

• gatling
• performance
• performance-testing
• gatling-plugin
• dsl
• feeders
• +6 more

32.milessabin / shapeless

Generic programming for Scala

• scala
• typelevel
• functional-programming
• generic-programming
• fp

33.sangria-graphql / sangria

Scala GraphQL implementation

• sangria
• graphql
• scala
• api

34.scalapb / ScalaPB

Protocol buffer compiler for Scala.

• scalapb
• protocol-buffers
• scala

35.ghostdogpr / caliban

Functional GraphQL library for Scala

• scala
• graphql
• graphql-server
• functional-programming
• graphql-client

36.twitter / util

Wonderful reusable code from Twitter

• finagle
• scala
• utility

37.monix / monix

Asynchronous, Reactive Programming for Scala and Scala.js.

• scala
• monix
• typelevel
• functional-programming
• reactive-programming
• fp
• +4 more

38.holdenk / spark-testing-base

Base classes to use when writing tests with Spark

39.softwaremill / quicklens

Modify deeply nested case class fields

• scala
• lenses
• functional-programming

40.scalatest / scalatest

A testing tool for Scala and Java developers

41.h2oai / sparkling-water

Sparkling Water provides H2O functionality inside Spark cluster

• h2o
• spark
• machine-learning
• integration
• pysparkling
• rsparkling
• +3 more

42.neo4j / neo4j-spark-connector

Neo4j Connector for Apache Spark, which provides bi-directional read/write access to Neo4j from Spark, using the Spark DataSource APIs

• spark
• neo4j-connector
• cypher
• neo4j-driver
• bolt
• hacktoberfest

43.twitter / scalding

A Scala API for Cascading

44.scalameta / munit

Scala testing library with actionable errors and extensible APIs

45.bkirwi / decline

A composable command-line parser for Scala.

46.archivesunleashed / aut

The Archives Unleashed Toolkit is an open-source toolkit for analyzing web archives.

• spark
• hadoop
• webarchives
• analysis
• apache-spark
• scala
• +8 more

47.ReactiveMongo / ReactiveMongo

:leaves: Non-blocking, Reactive MongoDB Driver for Scala

• scala
• reactivemongo
• mongodb
• database
• driver
• reactive
• +1 more

48.apache / incubator-toree

Mirror of Apache Toree (Incubating)

• toree

49.Hydrospheredata / mist

Serverless proxy for Spark cluster

• apache-spark
• api
• big-data
• serverless

50.twitter / summingbirdArchived

Streaming MapReduce with Scalding and Storm

51.databricks / spark-xmlArchived

XML data source for Spark SQL and DataFrames

52.twitter / storehaus

Storehaus is a library that makes it easy to work with asynchronous key value stores

53.Stratio / Spark-MongoDBArchived

Spark library for easy MongoDB access

54.lolgab / snunit

Scala Native HTTP server based on FreeUnit

• nginx-unit
• scala-native
• http-server
• scala
• serverless
• cask
• +3 more