Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
142,273 visible repositories in the index
1–24 of 26 · Most stars
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Distributed and fault-tolerant realtime computation: stream processing, continuous computation, distributed RPC, and more
A modular implementation of timely dataflow in Rust
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统
- graph
- graph-computation
- graph-neural-networks
- gremlin
The official repository for ROOT: analyzing, storing and visualizing big data, scientifically
- data-analysis
- root-cern
- root
- c-plus-plus
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
Distributed self-replicating programs in Python
- python
- devops
- infrastructure
- ansible
A cluster computing framework for processing large-scale geospatial data
- cluster-computing
- spatial-sql
- geospatial
- spatial-analysis
A new arguably faster implementation of Apache Spark from scratch in Rust
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.
- clickhouse
- simd
- spark-sql
- vectorization
parallel graph management and execution in heterogeneous computing
AI based entity resolution, identity resolution, master data management and deduplication at scale for humans and agents
- fuzzymatch
- fuzzy-matching
- deduplication
- dedupe
:chart_with_upwards_trend: Adaptive: parallel active learning of mathematical functions
- adaptive-learning
- machine-learning
- python
- live-plots
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
C implementation of the Raft Consensus protocol, BSD licensed
- raft-consensus-algorithm
Cloud-native genomic dataframes and batch computing
- genetics
- vcf
- genomics
- gwas
:rocket: R package: future: Unified Parallel and Distributed Processing in R for Everyone
- r
- cran
- parallel-processing
- parallel-computing
vineyard (v6d): an in-memory immutable data manager. (Project under CNCF, TAG-Storage)
- distributed-comp
- distributed-systems
- in-memory-storage
- big-data-analytics
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.
Distributed batch data processing framework