Apache Spark - A unified analytics engine for large-scale data processing
- python
- scala
- r
- java
Repository Index
81,588 visible repositories in the index
1–11 of 11 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统
- graph
- graph-computation
- graph-neural-networks
- gremlin
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
A cluster computing framework for processing large-scale geospatial data
- cluster-computing
- spatial-sql
- geospatial
- spatial-analysis
A new arguably faster implementation of Apache Spark from scratch in Rust
Phoebus is a distributed framework for large scale graph processing written in Erlang.
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake