Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
65,264 visible repositories in the index
1–24 of 37 · Most stars
CMAK is a tool for managing Apache Kafka clusters
Apache OpenWhisk is an open source serverless cloud platform
- openwhisk
- apache
- serverless
- cloud
Simple and Distributed Machine Learning Python Library porting ML algorithms for Spark
A machine learning package built for humans.
Deequ is a library built on top of Apache Spark for defining "unit tests for data", which measure data quality in large datasets.
- dataquality
- spark
- unit-testing
- scala
酷玩 Spark: Spark 源代码解析、Spark 类库等
- spark
- spark-streaming
- structured-streaming
- sparkcore
REST job server for Apache Spark
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
Apache Spark to Apache Cassandra connector
Build highly concurrent, distributed, and resilient message-driven applications using Java/Scala
- actor-model
- cloud-native
- concurrency
- distributed-actors
GraphFrames is a package for Apache Spark which provides DataFrame-based Graphs
- apache-spark
- big-data
- connected-components
- dataframe
ADAM is a genomics analysis platform with specialized file formats built using Apache Avro, Apache Spark, and Apache Parquet. Apache 2 licensed.
Project SnappyData - memory optimized analytics database, based on Apache Spark™ and Apache Geode™. Stream, Transact, Analyze, Predict in one cluster
Livy is an open source REST interface for interacting with Apache Spark from anywhere
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
uPickle: a simple, fast, dependency-free JSON & Binary (MessagePack) serialization library for Scala
Lightweight real-time big data streaming engine over Apache Pekko
- akka
- stream-processing
- bigdata
- scala
Mirror of Apache Toree (Incubating)
- toree
Low-code tool for automating actions on real time data | Stream processing for the users.
- flink
- flink-kafka
- apache-flink
- gui
[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark
The Apache Spark - Apache HBase Connector is a library to support Spark accessing HBase table as external data source or sink.
[unmaintained] Scala wrapper for Apache TinkerPop 3 Graph DSL
- gremlin
- scala
- graph-database
- graph
Apache Spark testing helpers (dependency free & works with Scalatest, uTest, and MUnit)