Infrastructure for continually self‑improving agents
Repository Index
132,241 visible repositories in the index
49–72 of 324 · Most stars
Translated💎1MB lightweight face detection model
Home of StarCoder: fine-tuning & inference!
Native MLX runtime for Laya typed decision models — 7–14 ms short decisions on M3 Max. No text generation, PyTorch, or cloud API.
- apple-silicon
- inference
- machine-learning
- mlx
[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
Modeling, training, eval, and inference code for OLMo
An anomaly detection library comprising state-of-the-art algorithms and features such as experiment management, hyper-parameter optimization, and edge inference.
- anomaly-localization
- anomaly-segmentation
- anomaly-detection
- unsupervised-learning
Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference - Red and Blue Teams
- python
- attack
- adversarial-machine-learning
- poisoning
Awesome autocompletion, static analysis and refactoring library for python
- static-analysis
- auto-complete
- python
- refactoring
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Uplift modeling and causal inference with machine learning algorithms
- incubation
- machine-learning
- causal-inference
- uplift-modeling
MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.
- llava
- llm
- mlx
- vision-transformer
🔥🔥🔥AidLearning is a powerful AIOT development platform, AidLearning builds a linux env supporting GUI, deep learning and visual IDE on Android...Now Aid supports CPU+GPU+NPU for inference with high performance acceleration...Linux on Android or HarmonyOS
learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen
- course
- large-language-model
- llm
- python
General plug-and-play inference library for Recursive Language Models (RLMs), supporting various sandboxes.
A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.
Framework agnostic sliced/tiled inference + interactive ui + error analysis plots
- object-detection
- instance-segmentation
- computer-vision
- small-object-detection
Inference and training library for high-quality TTS models.
📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉
- flash-attention
- paged-attention
- tensorrt-llm
- vllm
Superduper: End-to-end framework for building custom AI applications and agents.
- ai
- mlops
- torch
- transformers
State of the art inference performance with modern accelerators on Kubernetes
An easy to use PyTorch to TensorRT converter
- jetson-nano
- jetson-tx2
- jetson-xavier
- pytorch
A unified inference and post-training framework for accelerated video generation.
- diffusers
- diffusion-models
- video-generation
- distillation
AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.