Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Repository Index
47,133 repositories indexed
1–24 of 43 · Most stars
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Making large AI models cheaper, faster and more accessible
- deep-learning
- hpc
- large-scale
- data-parallelism
The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more
- pytorch
- resnet
- pretrained-models
- pretrained-weights
Distributed Task Queue (development branch)
- python
- task-manager
- task-scheduler
- task-runner
TranslatedChinese translation of 《Designing Data-Intensive Application》 (DDIA), 1st edition / 2nd edition
A hyperparameter optimization framework
Easy-to-use and powerful LLM and SLM library with awesome model zoo.
- nlp
- embedding
- bert
- ernie
The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.
- cloud-computing
- deep-learning
- gpu
- hyperparameter-tuning
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Modin: Scale your Pandas workflows by changing a single line of code
- dataframe
- pandas
- distributed
- datascience
Build, Manage and Deploy AI/ML Systems
- machine-learning
- model-management
- ai
- ml
Distributed Asynchronous Hyperparameter Optimization in Python
- hacktoberfest
Centralized network visibility and continuous asset discovery. Monitor devices, detect change, and stay aware across distributed networks.
- network-security
- selfhosted
- wifi-network
- wifi-security
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Real-time monitor and web admin for Celery distributed task queue
- celery
- task-queue
- monitoring
- administration
Python Stream Processing
- kafka-streams
- kafka
- python
- asyncio
Distributed Evolutionary Algorithms in Python
Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.
A fast and reliable background task processing library for Python 3.
- python
- task-manager
- task-scheduler
- task-runner
Superduper: End-to-end framework for building custom AI applications and agents.
- ai
- mlops
- torch
- transformers
OpenDILab Decision AI Engine. The Most Comprehensive Reinforcement Learning Framework B.P.
- reinforcement-learning
- multiagent-reinforcement-learning
- self-play
- imitation-learning
Accelerated deep learning R&D
Distributed task queue with full async support
- asyncio
- python
- python3
- task-executor