Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
73,362 visible repositories in the index
1–21 of 21 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Enterprise job scheduling middleware with distributed computing ability.
- scheduler
- workflow
- distributed
- mapreduce
Data-Centric Pipelines and Data Versioning
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.
- agents
- ai
- decentralized
- distributed
A modular implementation of timely dataflow in Rust
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
The official repository for ROOT: analyzing, storing and visualizing big data, scientifically
- data-analysis
- root-cern
- root
- c-plus-plus
Open MPI main development repository
- c
- hpc
- mpi
- fortran
Open-source software for volunteer computing and grid computing.
- boinc
- distributed-computing
- volunteer-computing
- high-performance-computing
A new arguably faster implementation of Apache Spark from scratch in Rust
rust raft with improvements
- consensus
- distributed-systems
- raft
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Essential Spark extensions and helper methods ✨😲
- spark
- dataframe
Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.
Parallel Workflow extension for Rake, runs on multicores, clusters, clouds.
- ruby
- rake
- pwrake
- parallel
Common Lisp distributed computation framework implemented using Swank Client