Nano vLLM
Repository Index
111,085 visible repositories in the index
25–48 of 315 · Most stars
Machine learning, in numpy
- machine-learning
- neural-networks
- topic-modeling
- gaussian-mixture-models
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
- llm
- llmops
- model-inference
- fine-tuning
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
The repository provides code for running inference and finetuning with the Meta Segment Anything Model 3 (SAM 3), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.
My Python scripts to make high-quality figures for publications in top AI conferences and journals.
- figures
- python
- icml
- nature-machine-intelligence
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
Spark-TTS Inference Code
🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
- bloom
- deep-learning
- distributed-systems
- language-models
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
- yolov3
- object-detection
- yolo
- deep-learning
Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.
- generative-ai
- ltx
- ltx-2
Bayesian Modeling and Probabilistic Programming in Python
- python
- statistical-analysis
- bayesian-inference
- mcmc
SANA & Sol: Efficient Generative Models and Full-Stack Training & Inference Engines for Image, Video, and World Modeling
- diffusion
- dit
- pytorch
- sana
Open-source inference platform for language, vision, embedding, reranking, image, audio, and video models. Deploy with multiple engines across heterogeneous hardware, from a laptop to a distributed cluster. Serve models through OpenAI-compatible APIs.
Easily fine-tune (SFT/RL), evaluate, and deploy Qwen, Gemma, or any open agentic LLM/VLM!
Deep universal probabilistic programming with Python and PyTorch
- python
- pytorch
- machine-learning
- bayesian
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
- model-serving
- mlops
- llmops
- generative-ai
AlphaFold 3 inference pipeline.
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
- chatgpt
- deep-learning
- instruction-following
- language-model
DoWhy is a Python library for causal inference that supports explicit modeling and testing of causal assumptions. DoWhy is based on a unified language for causal inference, combining causal graphical models and potential outcomes frameworks.
- causal-inference
- machine-learning
- graphical-models
- bayesian-networks
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
- cuda-kernels
- deepspeed
- fastertransformer
- llm-inference
TranslatedA collection of the Android SDK needed for Android development, tools used in development, Android development tutorials, Android design guidelines, free design resources, and more.
- android-sdk
- android-studio
- sdk-manager
- android-development