Translated✍🏻 Source Code Deep Dives, System Design & Engineering Blogs | Halfrost-Field (冰霜之地, "Land of Frost"): notes on source code analysis, system design, and engineering practice
- ios
- objective-c
- swift
- javascript
Repository Index
47,133 repositories indexed
49–72 of 167 · Most stars
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
Open source alternative to AWS. Elastic compute, block storage (non replicated), firewall and load balancer, managed Postgres, K8s, AI inference, and IAM services.
- cloud
- portable
- bare-metal
- hosting
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
The repository provides code for running inference and finetuning with the Meta Segment Anything Model 3 (SAM 3), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
vCluster creates tenant clusters: fully isolated environments delivered as managed Kubernetes, or as the foundation for Slurm, Ray, Run:ai and inference clusters. Each gets its own API server, CRDs and RBAC, and runs on an existing cluster or standalone on bare metal. CNCF Certified Kubernetes.
- kubernetes
- vcluster
- virtual-clusters
- multi-tenancy
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
- inference
- deep-learning
- openvino
- ai
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
Spark-TTS Inference Code
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
- distributed-file-system
Production ready toolkit to run AI locally
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
Bayesian Modeling and Probabilistic Programming in Python
- python
- statistical-analysis
- bayesian-inference
- mcmc
Deep universal probabilistic programming with Python and PyTorch
- python
- pytorch
- machine-learning
- bayesian
The modular and type safe schema library for validating structural data 🤖
- bundle-size
- modular
- parsing
- schema
AlphaFold 3 inference pipeline.
Infrastructure for continually self‑improving agents
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
Awesome autocompletion, static analysis and refactoring library for python
- static-analysis
- auto-complete
- python
- refactoring
Uplift modeling and causal inference with machine learning algorithms
- incubation
- machine-learning
- causal-inference
- uplift-modeling