Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
67,262 visible repositories in the index
1–24 of 49 · Most stars
CMAK is a tool for managing Apache Kafka clusters
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
ZIO — A type-safe, composable library for async and concurrent programming in Scala
- effects
- concurrency
- asynchronicity
- functional-programming
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
Open-source code analysis platform for C/C++/Java/Binary/Javascript/Python/Kotlin based on code property graphs. Discord https://discord.gg/vv4MH284Hc
- code-analysis
- query-language
- code-browser
- fuzzy-parsing
In-memory dimensional time series database.
Spark: The Definitive Guide's Code Repository
Slick (Scala Language Integrated Connection Kit) is a modern database query and access library for Scala
A Scala API for Apache Beam and Google Cloud Dataflow.
- scala
- bigquery
- google-cloud
- beam
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
A data access library for Scala + Postgres.
- scala
- postgres
- functional-programming
- cats
MLeap: Deploy ML Pipelines to Production
- scikit-learn
- spark
- data-pipelines
- transformers
Distributed Prometheus time series database
- scala
- time-series
- database
- distributed
Scala library for boilerplate-free, type-safe data transformations
- scala
- boilerplate
- generic-programming
- automatic-conversion
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster
Sparkling Water provides H2O functionality inside Spark cluster
- h2o
- spark
- machine-learning
- integration
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
CPU and GPU-accelerated Machine Learning Library
:leaves: Non-blocking, Reactive MongoDB Driver for Scala