Executable file for VITS inference
Repository Index
132,241 visible repositories in the index
121–144 of 324 · Most stars
An agentic company research tool powered by LangGraph and Tavily that conducts deep diligence on companies using a multi-agent framework. It leverages OpenRouter on the backend for rapid inference with any leading model.
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.
- machine-learning
- python
- gpu
- llms
Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.
micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape
- quantization
- pruning
- dorefa
- twn
Communicate with an LLM provider using a single interface
Code, labs, and resources for O'Reilly AI Systems Performance Engineering: GPU optimization, distributed training, inference scaling, and full-stack tuning.
推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.
- calibration
- causal-inference
- cold-start
- contrastive-learning
A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow
- bayesian-inference
- probabilistic-programming
- graphical-models
- generative-models
MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.
Adaptive Test-time Learning and Autonomous Specialization
- ai-agent
- ai-harness
- coding-agent
- developer-tools
Open Source AI Accelerator Research Platform Standard / 开源推理研究平台
Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond
- gpu-sharing
- kvcache
- llm
- sglang
14-stage Fusion Pipeline for LLM token compression — reversible compression, AST-aware code analysis, intelligent content routing. Zero LLM inference cost. MIT licensed.
- ai-agent-tools
- context-compression
- context-pruning
- context-window-optimization
Gaussian processes in TensorFlow
- gaussian-processes
- tensorflow
- gpflow
- machine-learning
Large-scale LLM inference engine
- api-rest
- inference-engine
- machine-learning
- cuda
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
- function-calling
- llm
- llm-agent
- llm-agents
Tile-Based Runtime for Ultra-Low-Latency LLM Inference
基于PaddleOCR重构,并且脱离PaddlePaddle深度学习训练框架的轻量级OCR,推理速度超快 —— A lightweight OCR system based on PaddleOCR, decoupled from the PaddlePaddle deep learning training framework, with ultra-fast inference speed.
- document-layout-analysis
- license-plate-recognition
- ocr
- onnx-models
TensorFlow template application for deep learning
- tensorflow
- tfrecords
- libsvm
- csv
GPT-SoVITS ONNX Inference Engine & Model Converter
- gpt-sovits
- tts
- text-to-speech
- vits
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
High Performance Serving Runtime For Interactive Super Intelligence System
TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration