Trending Tags Repositories Insights Dependencies Withheld

/

Trending Tags Repositories Insights Dependencies Withheld

Trending repositories

Ranked by Attention Score: stars gained over the last 30 complete days

Period

Sort order

Topics: tokenizer

Topics

Search topics

No listed topic matches. Press Enter to filter by “”.

Tag

Filters

Apply

Filters

Primary language

License

Total stars

Last push

Created

Archived

Clear all filters

4 repositories by Attention Score · Oct 4, 2026 – Nov 2, 2026 (UTC)

Topic: tokenizer

1.rasbt / LLMs-from-scratch

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

gpt
large-language-models
llm
python
pytorch
ai
+13 more

Jupyter Notebook Other license108K total17K forks

+1,835 stars

from 105,815 · score 7.52

2.Chevrotain / chevrotain

Parser Building Toolkit for JavaScript

javascript
typescript
parser-library
parsing
grammars
tokenizer
+2 more

TypeScript APACHE-2.02.8K total222 forks

+11 stars

from 2,805 · score 2.48

3.ikawaha / kagome

Self-contained Japanese Morphological Analyzer written in pure Go

japanese
tokenizer
nlp-library
japanese-language
pos-tagging
segmentation
+3 more

Go MIT987 total61 forks

+4 stars

from 983 · score 1.61

4.andialbrecht / sqlparse

A non-validating SQL parser module for Python

formatter
python
sql
sql-formatter
sql-parser
tokenizer

Python BSD-3-CLAUSE4K total778 forks

+1 stars

from 4,021 · score 0.69

Order: Attention Score

Most net stars gained in the window first. The score only puts this gain on a log scale, so the order is the order of the gain.

How the ranking works

Ranked by stars gained in the window. The score is the natural log of one plus that gain, so a repository's rank is exactly its star gain's rank.

score = log1p(net stars gained in the trailing T complete UTC days)

A repository is ranked only when its Star History covers every day of the window and its latest Observation is not missing.

The longest window is 180 days. Star History from the Repository Host goes back at most 180 days before a repository's first Observation. A longer window needs more days of Observations first. A repository younger than the window is not ranked in it.

The filters narrow the same eligible repositories. A repository matches the topics when it has any of them. Total stars is the count now. Last push and creation on the Repository Host count back from the start of today (UTC). The counts beside the languages, licenses, and topics include all ranked repositories in the window. A Tag holds the repositories that have it; see Tags.

Repositories enter the index from the Seed Dataset or by gaining stars faster than the published admission threshold, whatever their size. Admission is permanent, so an archived repository can still rank.

Browse the Repository Index

Trending Tags Repositories Insights Dependencies Withheld Withholding Rule Health facts Privacy Send feedback

Powered by Workhorse · From the makers of AwesomeRepos