100 days of LLM inference engineering — daily posts, experiments, and visualizations
Repository Index
111,821 visible repositories in the index
385–408 of 872 · Most stars
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere (Linux/Windows/Mac). Use `llama2-wrapper` as your local llama2 backend for Generative Agents/Apps.
- llama-2
- llama2
- llm
- llm-inference
Gaussian processes in TensorFlow
- gaussian-processes
- tensorflow
- gpflow
- machine-learning
Publications from Trail of Bits
- conference-presentations
- academic-papers
- security-reviews
Large-scale LLM inference engine
- api-rest
- inference-engine
- machine-learning
- cuda
[ICML 2024] LLMCompiler: An LLM Compiler for Parallel Function Calling
- function-calling
- llm
- llm-agent
- llm-agents
Tile-Based Runtime for Ultra-Low-Latency LLM Inference
Group peer to peer video calls for everyone written in Go and TypeScript
- peer-calls
- peer-to-peer
- p2p
- video-call
🚀 Easier & Faster YOLO Deployment Toolkit for NVIDIA 🛠️
- tensorrt
- tensorrt-inference
- tensorrt10
- computer-vision
Awesome AI Learning with +100 AI Cheat-Sheets, Free online Books, Top Courses, Best Videos and Lectures, Papers, Tutorials, +99 Researchers, Premium Websites, +121 Datasets, Conferences, Frameworks, Tools
Mathematical formulations and circuit designs for zero-knowledge proofs.
基于PaddleOCR重构,并且脱离PaddlePaddle深度学习训练框架的轻量级OCR,推理速度超快 —— A lightweight OCR system based on PaddleOCR, decoupled from the PaddlePaddle deep learning training framework, with ultra-fast inference speed.
- document-layout-analysis
- license-plate-recognition
- ocr
- onnx-models
将视频瞬间转化为手绘故事 Turn Video Moments into Hand-Drawn Stories
The non profit fostering AI in India through credits, grants, resources
- ai
- grants
- hackathons
- llm-inference
TensorFlow template application for deep learning
- tensorflow
- tfrecords
- libsvm
- csv
Unified, efficient, and easy-to-use inference infrastructure for edge AI || 面向端侧 AI 的统一、高效、易用的推理基础设施
The Offensive Manual Web Application Penetration Testing Framework.
- web-penetration-testing
- reconnaissance
- vulnerability-analysis
- scanning-enumeration
An R package for causal inference in time series
A general-purpose probabilistic programming system with programmable inference
- probabilistic-programming
- gen
- deep-learning
- bayesian
Ultrafast serverless GPU inference, sandboxes, and background jobs
- gpu
- ml-platform
- cuda
- fine-tuning
GPT-SoVITS ONNX Inference Engine & Model Converter
- gpt-sovits
- tts
- text-to-speech
- vits
The inference module for AgiBot X1.
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration