Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
47,133 repositories indexed
1–24 of 42 · Most stars
CMAK is a tool for managing Apache Kafka clusters
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
The leader in Customer Data Infrastructure
- snowplow
- analytics
- data
- data-pipeline
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
In-memory dimensional time series database.
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
A data access library for Scala + Postgres.
- scala
- postgres
- functional-programming
- cats
MLeap: Deploy ML Pipelines to Production
- scikit-learn
- spark
- data-pipelines
- transformers
Distributed Prometheus time series database
Scala library for boilerplate-free, type-safe data transformations
- scala
- boilerplate
- generic-programming
- automatic-conversion
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster
Sparkling Water provides H2O functionality inside Spark cluster
- h2o
- spark
- machine-learning
- integration
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
CPU and GPU-accelerated Machine Learning Library
:leaves: Non-blocking, Reactive MongoDB Driver for Scala
Essential Spark extensions and helper methods ✨😲
- spark
- dataframe
Lightweight real-time big data streaming engine over Apache Pekko
- akka
- stream-processing
- bigdata
- scala
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark
BlinkDB: Sub-Second Approximate Queries on Very Large Data.