Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Repository Index
88,573 visible repositories in the index
1–20 of 20 · Most stars
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale
Distributed AI/LLM for the people. Share compute privately or publicly to power your agents and chat.
- agents
- ai
- decentralized
- distributed
Fast, efficient, and scalable distributed map/reduce system, DAG execution, in memory or on disk, written in pure Go, runs standalone or distributedly.
- distributed-computing
- map-reduce
- golang
- distributed-systems
🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba | 一站式图计算系统
- graph
- graph-computation
- graph-neural-networks
- gremlin
Glow is an easy-to-use distributed computation system written in Go, similar to Hadoop Map Reduce, Spark, Flink, Storm, etc. I am also working on another similar pure Go system, https://github.com/chrislusf/gleam , which is more flexible and more performant.
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.
Distributed self-replicating programs in Python
- python
- devops
- infrastructure
- ansible
Open-source software for volunteer computing and grid computing.
- boinc
- distributed-computing
- volunteer-computing
- high-performance-computing
YTsaurus is a scalable and fault-tolerant open-source big data platform.
- big-data
- clickhouse
- distributed-database
- lakehouse
Apache Hadoop docker image
- hadoop-docker
- docker-hadoop
- hadoop
- hadoop-cluster
Apache Spark docker image
- spark-kubernetes
- kubernetes
- k8s-spark
- docker
Feathr – A scalable, unified data and AI engineering platform for enterprise
- feature-engineering
- feature-store
- artificial-intelligence
- mlops
Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.
Python 3.8+ toolbox for submitting jobs to Slurm
- slurm
- python
- clusters
Map-Reduce for Clojure
Open Lakehouse Compute Fabric: an open-source lakehouse platform with Microsoft Fabric Compute Extension.
- apache-iceberg
- apache-spark
- data-engineering
- delta-lake
Standard API for Distributed Data Structures in R