1.rasbt / LLMs-from-scratchImplement a ChatGPT-like LLM in PyTorch from scratch, step by stepgptlarge-language-modelsllmpythonpytorchai+13 moreJupyter NotebookOther license108K total17K forks+7,617 starsfrom 100,033 · score 8.94
2.Chevrotain / chevrotainParser Building Toolkit for JavaScriptjavascripttypescriptparser-libraryparsinggrammarstokenizer+2 moreTypeScriptAPACHE-2.02.8K total222 forks+24 starsfrom 2,792 · score 3.22
3.andialbrecht / sqlparseA non-validating SQL parser module for Pythonformatterpythonsqlsql-formattersql-parsertokenizerPythonBSD-3-CLAUSE4K total778 forks+17 starsfrom 4,005 · score 2.89
4.ikawaha / kagomeSelf-contained Japanese Morphological Analyzer written in pure Gojapanesetokenizernlp-libraryjapanese-languagepos-taggingsegmentation+3 moreGoMIT987 total61 forks+12 starsfrom 975 · score 2.56
5.bzick / tokenizerTokenizer (lexer) for anythinggolangtokenizerparsetokenizinglexerparserGoMIT142 total11 forks+1 starsfrom 141 · score 0.69
6.mamantoha / toon-crystalCrystal implementation of the Token-Oriented Object Notation(TOON) serialization formatdata-formatllmserializationtokenizerCrystalMIT10 total0 forks+1 starsfrom 9 · score 0.69