Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
108,094 visible repositories in the index
1–24 of 25 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Data-Centric Pipelines and Data Versioning
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Apache Ignite
- distributed-sql-database
- iot
- osgi
- network-client
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统
- graph
- graph-computation
- graph-neural-networks
- gremlin
Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.
Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.
The official repository for ROOT: analyzing, storing and visualizing big data, scientifically
- data-analysis
- root-cern
- root
- c-plus-plus
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
💎多语言实现,高性能生成唯一数字ID。 💎优化的雪花算法(SnowFlake)——雪花漂移算法,在缩短ID长度的同时,具备极高瞬时并发处理能力(50W/0.1s)。 💎原生支持 C#/Java/Go/Rust/C/JavaScript/TypeScript/Python/Pascal 多语言,提供其它适用于其它语言的多线程安全调用动态库(FFI)。💎支持容器环境自动扩容(自动注册 WorkerId ),单机或分布式唯一IdGenerator。💎顶尖优化,超强效能。
- snowflake
- idgenerator
- dotnet
- rust
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
nebulai's depin computing services
Apache Spark docker image
- spark-kubernetes
- kubernetes
- k8s-spark
- docker
Feathr – A scalable, unified data and AI engineering platform for enterprise
- feature-engineering
- feature-store
- artificial-intelligence
- mlops
parallel graph management and execution in heterogeneous computing
Apache DataFusion Comet Spark Accelerator
- arrow
- datafusion
- rust
- spark
Distribute and run AI workloads on Kubernetes magically in Python, like PyTorch for ML infra.
Making data lake work for time series
- data-lake-analytics
- distributed
- etl-framework
- mlops
Cloud-native genomic dataframes and batch computing
- genetics
- vcf
- genomics
- gwas
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Essential Spark extensions and helper methods ✨😲
- spark
- dataframe
Map-Reduce for Clojure