A lightweight inference engine supporting speculative speculative decoding (SSD).
Repository Index
132,241 visible repositories in the index
217–240 of 324 · Most stars
Python package for point cloud registration using probabilistic model (Coherent Point Drift, GMMReg, SVR, GMMTree, FilterReg, Bayesian CPD)
- point-cloud
- point-cloud-registration
- coherent-point-drift
- gaussian-mixture-models
“AI-Compass”将为社区指引在 AI 技术海洋中航行的方向,无论你是初学者还是进阶开发者,都能在这里找到通往 AI 各大方向的路径。旨在帮助开发者系统性地了解 AI 的核心概念、主流技术、前沿趋势,并通过实践掌握从理论到落地的全过程。
Official inference code for Krea 2
Running Llama 2 and other Open-Source LLMs on CPU Inference Locally for Document Q&A
- cpu
- cpu-inference
- deep-learning
- faiss
Code for paper " AdderNet: Do We Really Need Multiplications in Deep Learning?"
- pytorch
- imagenet
- convolutional-neural-networks
- cvpr2020
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
- speculative-decoding
Official inference library for pre-processing of Mistral models
PyTorch implementation of normalizing flow models
- normalizing-flow
- variational-autoencoder
- variational-inference
- real-nvp
A CLI to estimate inference memory requirements for Hugging Face models, written in Python.
- huggingface
- safetensors
- gguf
- hf-extension
CausalVLR: A Toolbox and Benchmark for Vision-Language Causal Reasoning (多模态因果推理开源框架)
- causal
- causal-inference
- visual-linguistic
- multimodal
Real-time multi-person action recognition from OpenPose skeleton sequences, with tracking, feature extraction, model training, and inference pipelines for research and prototyping.
- action-recognition
- openpose
- machine-learning
- time-series
Inference code for the paper "Spirit-LM Interleaved Spoken and Written Language Model".
Fast inference from large lauguage models via speculative decoding
Fast and accurate face landmark detection library using PyTorch; Support 68-point semi-frontal and 39-point profile landmark detection; Support both coordinate-based and heatmap-based inference; Up to 100 FPS landmark inference speed with SOTA face detector on CPU.
- face-alignment
- face-landmark-detection
- onnx
- pytorch-face-landmark
inferring sql queries from plain-text questions about tables
- sql
- wikisql
- deep-learning
- natural-language
NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.
- deep-learning
- end-to-end
- gpu-acceleration
- machine-learning
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
- machine-learning
- scikit-learn
- xgboost
- lightgbm
LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.
- llm
- llm-inference
- triton-kernels
- vllm
AIPerf is a comprehensive benchmarking tool that measures the performance of generative AI models served by your preferred inference solution.
sbi is a Python package for simulation-based inference, designed to meet the needs of both researchers and practitioners. Whether you need fine-grained control or an easy-to-use interface, sbi has you covered.
- simulation-based-inference
- likelihood-free-inference
- bayesian-inference
- parameter-estimation
PyTorch Implementation of [1611.06440] Pruning Convolutional Neural Networks for Resource Efficient Inference
- deep-learning
- pruning
- pytorch
Phylogenetic orthology inference for comparative genomics
- gene-trees
- orthogroup
- orthologs
- gene-duplication
This repository contains tutorials and examples for Triton Inference Server