Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
- llama-2
- llama2
- llm
- llm-inference
132,241 visible repositories in the index
289–312 of 748 · Most stars
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
Gaussian processes in TensorFlow
[ICLR 2025] CatVTON is a simple and efficient virtual try-on diffusion model with 1) Lightweight Network (899.06M parameters totally), 2) Parameter-Efficient Training (49.57M parameters trainable) and 3) Simplified Inference (< 8G VRAM for 1024X768 resolution).
Large-scale LLM inference engine
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
Tile-Based Runtime for Ultra-Low-Latency LLM Inference
🚀 Easier & Faster YOLO Deployment Toolkit for NVIDIA 🛠️
Mathematical formulations and circuit designs for zero-knowledge proofs.
The non profit fostering AI in India through credits, grants, resources
基于PaddleOCR重构,并且脱离PaddlePaddle深度学习训练框架的轻量级OCR,推理速度超快 —— A lightweight OCR system based on PaddleOCR, decoupled from the PaddlePaddle deep learning training framework, with ultra-fast inference speed.
TensorFlow template application for deep learning
Unified, efficient, and easy-to-use inference infrastructure for edge AI || 面向端侧 AI 的统一、高效、易用的推理基础设施
An R package for causal inference in time series
A general-purpose probabilistic programming system with programmable inference
Ultrafast serverless GPU inference, sandboxes, and background jobs
GPT-SoVITS ONNX Inference Engine & Model Converter
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
The inference module for AgiBot X1.
High Performance Serving Runtime For Interactive Super Intelligence System
TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration
Vitis AI is Xilinx’s development stack for AI inference on Xilinx hardware platforms, including both edge devices and Alveo cards.
Proof of Inference - Trustless AI verification
Let's simplify the way we write Lisp
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs