tiktoken is a fast BPE tokeniser for use with OpenAI's models.
Repository Index
69,734 visible repositories in the index
1–22 of 22 · Most stars
Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.
A curated list of Web Performance Optimization.
- awesome-list
- awesome
- list
- webperformance
Automate running Lighthouse for every commit, viewing the changes, and preventing regressions
- lighthouse
- webperf
sitespeed.io is an open-source tool for comprehensive web performance analysis, enabling you to test, monitor, and optimize your website’s speed using real browsers in various environments.
- web
- metrics
- monitoring
- webperf
Scans a given process. Recognizes and dumps a variety of potentially malicious implants (replaced/injected PEs, shellcodes, hooks, in-memory patches).
- pe-format
- hooking
- pe-dumper
- pe-analyzer
Make optimisation of images part of your automated build process
- imageoptim
- image-optimisation
- jpegmini
- pngquant
"SQIP" (pronounced \skwɪb\ like the non-magical folk of magical descent) is a SVG-based LQIP technique.
- webperf
- webperformance
- performance
- image
Motion imitation with deep reinforcement learning.
All In One Web Recon
- python3
- pentesting
- pentest-tool
- pentesting-tools
WebPerf and front-end quality testing tool
A lightweight suite of motion imitation methods for training controllers.
- animation
- reinforcement-learning
- robotics
- motion-imitation
Unsupervised Word Segmentation for Neural Machine Translation and Text Generation
- neural-machine-translation
- segmentation
- machine-translation
- nmt
Headless Chromium-based web performance metrics collector and monitoring tool
Automating Web Performance testing with Puppeteer 🎪
Progressive web metrics at your fingertipz
- metrics
- performance
- webperf
- lighthouse
Performance-Bookmarklet helps to analyze the current page through the Resource Timing API, Navigation Timing API and User-Timing - requests by type, domain, load times, marks and more. Sort of a light live WebPageTest.
- webperf
- bookmarklet
- navigation-timing
- resource-timing
Dynamic unpacker based on PE-sieve
- libpeconv
- pe-sieve
- memory-forensics
- malware-unpacker
CSS selectors complexity and performance analyzer
Fast command-line HTML minifier — works on any HTML, with special support for Razor and Web Forms views. Parallel processing, safe preservation of <pre>/<code>/Knockout blocks, and easy MSBuild/CI integration.
- c-sharp
- html-minifier
- razor-views
- knockout
Low-overhead sampling profiler and tracer for Ruby for Linux
- ruby
- bpf
- performance
- profilers
PEG parser generator for VB6
- vb6
- peg
- parser
- grammar