A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations
Repository Index
105,102 visible repositories in the index
1–24 of 26 · Most stars
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.
An up-to-date (unofficial) knowledge base for Go programming self learning
- go
- golang
- book
- syntax
Fast and efficient CSS optimizer for node.js and the Web
- css
- minification
- optimization
- css-optimizations
CSS minifier with structural optimizations
- css-minifier
- css
- minifier
- usage-data
HElib is an open-source software library that implements homomorphic encryption. It supports the BGV scheme with bootstrapping and the Approximate Number CKKS scheme. HElib also includes optimizations for efficient homomorphic evaluation, focusing on effective use of ciphertext packing techniques and on the Gentry-Halevi-Smart optimizations.
- helib
- privacy-enhancing-technologies
- cryptography
- privacy-by-design
In order to make it easier to use the ComfyUI, I have made some optimizations and integrations to some commonly used nodes.
zlib replacement with optimizations for "next generation" systems.
- compression
- library
- deflate
- inflate
Roaring bitmaps in C (and C++), with SIMD (AVX2, AVX-512 and NEON) optimizations: used by Apache Doris, ClickHouse, Alibaba Tair, Redpanda, YDB and StarRocks
- roaring-bitmaps
- visual-studio
- c
- clang
Coding Agent singularly focused efficiency and context curation. Reduces API costs by 50-80% vs other agent AND improves the code quality at the same time. Uses Hash Anchored edits, massively parallel operations, AST manipulation and many many other optimizations. https://dirac.run/
Firefox fork with compiler optimizations and patches from Librewolf, Waterfox, and GNU IceCat.
- browser
- firefox
- mercury
- optimization
PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.
Tips and tricks to optimize your C++ code
- cpp
- optimization
- blog
Fast Compiler for C# Expression Trees and the lightweight LightExpression alternative. Diagnostic and code generation tools for the expressions.
- expression-tree
- dryioc
- delegate
- il-optimizations
This is a series of GPU optimization topics. Here we will introduce how to optimize the CUDA kernel in detail. I will introduce several basic kernel optimizations, including: elementwise, reduce, sgemv, sgemm, etc. The performance of these kernels is basically at or near the theoretical limit.
- gpu-acceleration
- elementwise
- reduce
- sgemm
https://wavespeed.ai/ Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.
- cuda
- diffusers
- pytorch
- stable-diffusion
Automatic verification of LLVM optimizations
- llvm
- verification
- translation-validation
- automatic-verification
Best resources for Solidity gas optimizations ⛽
- ethereum
- polygon
- ethereum-blockchain
- smart-contracts
⏱️ Investigates v8/Node.js function deoptimizations.
CUDA Tile IR is an MLIR-based intermediate representation and compiler infrastructure for CUDA kernel optimization, focusing on tile-based computation patterns and optimizations targeting NVIDIA tensor core units.
A Webpack boilerplate for static websites that has all the necessary modern tools and optimizations built-in. Score a perfect 10/10 on performance.
- webpack
- webpack-boilerplate
- optimization
- babel
Cryptocurrency trading bot with a graphical user interface with support for simulations, backtests, optimizations, and running live bots.
- simulations
- trade
- telegram
- cryptocurrency
Pure Go production-grade regex engine with SIMD optimizations. Up to 3-3000x+ faster than stdlib.
- avx2
- dfa
- go
- golang
A list of general purpose tricks and techniques for saving tokens in PICO-8.
- pico-8
- optimization-methods