Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
Repository Index
47,133 repositories indexed
1–18 of 18 · Most stars
Faster Whisper transcription with CTranslate2
- deep-learning
- inference
- quantization
- speech-recognition
中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)
- llm
- plm
- pre-trained-language-models
- alpaca
[🔥updating ...] AI 自动量化交易机器人(完全本地部署) AI-powered Quantitative Investment Research Platform. 📃 online docs: https://ufund-me.github.io/Qbot ✨ :news: qbot-mini: https://github.com/Charmve/iQuant
- funds
- machine-learning
- pytrade
- quantitative-finance
A vector index built on TurboQuant, written in Rust with Python bindings
- ann
- avx512
- embeddings
- faiss
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
- avx2
- c99
- cpu-inference
- deep-learning
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniMax-M3.1-Flash (428B MoE) privately on your own GPU(s), from one card to 16, NVIDIA or AMD, Linux or Windows. Engine, server, dashboard, OpenAI- and Anthropic-compatible API, quant tools.
Dataflow compiler for QNN inference on FPGAs
QKeras: a quantization deep learning library for Tensorflow Keras
- deep-learning
- quantization
- quantized-neural-networks
- hardware-acceleration
Smaller images, same picture. Compress PNGs to PNG or WebP with automatic quality. PNG output is 8–12% smaller than TinyPNG at equal quality on the median image. Command line and browser.
Official library for RaBitQ vector quantization and approximate nearest-neighbor search, in C++ and Python.
- nearest-neighbor-search
- quantization
- vector-database
- vector-search
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
- calibration
- cuda
- cuda-graphs
- decision-making
This repository contains notebooks that show the usage of TensorFlow Lite for quantizing deep neural networks.
- tensorflow-2
- tensorflow-lite
- on-device-ml
- model-quantization
🐋 Полный гайд на русском: как установить и запустить DeepSeek локально (V4, R1) на ПК, Mac и сервере — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, агенты, fine-tuning, лучшие фишки
- ai-agents
- deepseek
- deepseek-r1
- deepseek-v4
PrismQuant: Optimal Null-Space Rotations for Grouped Quantizers
- int4
- large-language-models
- pytorch
- quantization
Legend of Elya — N64 game with a real 6.36M-parameter ternary transformer on the VR4300 MIPS III CPU. Zelda-style dungeon, AI NPCs, byte-level inference at 1.23 tok/s scalar / 2.19 tok/s on the RSP overlay (measured under ares, never on silicon). Built with libdragon.
Color quantization and dithering in TypeScript.