Learn how to design systems at scale and prepare for system design interviews
- architecture
- distributed-systems
- system-design
- system-design-interview
Repository Index
47,133 repositories indexed
1–24 of 42 · Most stars
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Neon: Serverless Postgres. We separated storage and compute to offer autoscaling, code-like database branching, and scale to zero.
A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.
- gbdt
- gbm
- machine-learning
- data-mining
Paradev (formerly Superset) is an agentic IDE to orchestrate 100+ coding agents in parallel. Run any agent with your own subscription.
A hyperparameter optimization framework
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
Self-hostable workflow orchestrator for teams whose main work isn't orchestration. Declarative YAML over your scripts, SSH commands, containers, etc; keep workflows separate from business logic. One binary, no database, runs on limited H/W resources. Alternative to Airflow / Cron / Job Scheduler.
- cron
- workflow-engine
- workflow-scheduler
- workflow-orchestration
Reliable PostgreSQL Backup & Restore
- postgresql
- parallel
- backup
- restore
Packer templates for building minimal Vagrant baseboxes for multiple platforms
- packer-templates
- vagrant
- parallels
- vmware
Parallel S3 and local filesystem execution tool.
AI Infra / AI Orchestration / AI Control Plane
Determined is an open-source machine learning platform that simplifies distributed training, hyperparameter tuning, experiment tracking, and resource management. Works with PyTorch and TensorFlow.
- deep-learning
- machine-learning
- ml-platform
- ml-infrastructure
A family of header-only, very fast and memory-friendly hashmap and btree containers.
- hash
- tables
- unordered-map
- unordered-set
The C++ Standard Library for Parallelism and Concurrency
- c-plus-plus
- distributed
- cross-platform
- parallelism
Kubernetes-native platform to run massively parallel data/streaming jobs
- kubernetes
- stream-processing
- data-processing
- k8s
Singularity has been renamed to Apptainer as part of us moving the project to the Linux Foundation. This repo has been persisted as a snapshot right before the changes.
- containers
- container
- singularity-container
- portable
Fastest end-to-end CSV ingestion for Ruby (with C acceleration). SmarterCSV auto-detects formats, applies smart defaults, and returns Rails-ready hashes for seamless use with ActiveRecord, Sidekiq, parallel jobs, and S3 pipelines — even for messy user-uploaded real-world data.
- csv
- csv-converter
- csv-export
- csv-file
Native speech-to-text for Linux - Fast, accurate, private, and hackable system-wide dictation
AWS ParallelCluster is an AWS supported Open Source cluster management tool to deploy and manage HPC clusters in the AWS cloud.
Pythonic tool for orchestrating machine-learning/high performance/quantum-computing workflows in heterogeneous compute environments.
- hpc
- workflow
- workflow-automation
- workflow-management
Multitenant backend server for building web and mobile apps rapidly. The backend for busy developers. (self-hosted or hosted)
- backend
- api-server
- json-api
- baas
Multi-agent orchestration platform for Gemini CLI, Claude Code, Codex, and Qwen Code — 39 specialists, parallel subagents, persistent sessions, and built-in code review, debugging, security, SEO, accessibility, and compliance tools
Install Tiledesk on your server using Helm for Kubernetes orchestration and Docker Compose for running multi-container Docker applications. Tiledesk provides an open-source solution comparable to Voiceflow, empowering you to create sophisticated LLM-enabled chatbots that seamlessly transition interactions to human agents when needed.
- kubernetes
- docker
- helm-charts
- installation