Sign in

Repository Index

132,241 visible repositories in the index

Tags: All Tags
Tags

A repository shows when it has any selected Tag.

Filters (1)

Filters

Clear all filters

121–144 of 324 · Most stars

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

  • quantization
  • pruning
  • dorefa
  • twn

推荐/广告/搜索领域工业界经典以及最前沿论文集合。A collection of industry classics and cutting-edge papers in the field of recommendation/advertising/search.

  • calibration
  • causal-inference
  • cold-start
  • contrastive-learning

A probabilistic programming library for Bayesian deep learning, generative models, based on Tensorflow

  • bayesian-inference
  • probabilistic-programming
  • graphical-models
  • generative-models

14-stage Fusion Pipeline for LLM token compression — reversible compression, AST-aware code analysis, intelligent content routing. Zero LLM inference cost. MIT licensed.

  • ai-agent-tools
  • context-compression
  • context-pruning
  • context-window-optimization

基于PaddleOCR重构,并且脱离PaddlePaddle深度学习训练框架的轻量级OCR,推理速度超快 —— A lightweight OCR system based on PaddleOCR, decoupled from the PaddlePaddle deep learning training framework, with ultra-fast inference speed.

  • document-layout-analysis
  • license-plate-recognition
  • ocr
  • onnx-models

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration