Find the ghost tokens. Fix them. Survive compaction. Avoid context quality decay.
- claude-code
- token-optimization
- context-window
- token-usage
109,091 visible repositories in the index
25–48 of 87 · Most stars
Find the ghost tokens. Fix them. Survive compaction. Avoid context quality decay.
LinkToken is an open-source unified token management platform that provides token distribution, access control, and usage analysis functions.
TokenSpeed is a speed-of-light LLM inference engine.
IoT Platform Framework
Local typed decisions, contrastive data curation, and model evaluation.
Local-first AI token usage & cost tracker for 31 coding tools incl. Claude Code, Codex, Cursor, Gemini & DeepSeek Harness—with native apps. Never reads prompts.
Specify a github or local repo, github pull request, arXiv or Sci-Hub paper, Youtube transcript or documentation URL on the web and scrape into a text file and clipboard for easier LLM ingestion
14-stage Fusion Pipeline for LLM token compression — reversible compression, AST-aware code analysis, intelligent content routing. Zero LLM inference cost. MIT licensed.
Easy token price estimates for 400+ LLMs. TokenOps.
Synthetic data curation for post-training and structured data extraction
中文对话0.2B小模型(ChatLM-Chinese-0.2B),开源所有数据集来源、数据清洗、tokenizer训练、模型预训练、SFT指令微调、RLHF优化等流程的全部代码。支持下游任务sft微调,给出三元组信息抽取微调示例。
Online playground for OpenAPI tokenizers
NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness.
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
The repository host gives no description.
Colivara is a suite of services that allows you to store, search, and retrieve documents based on their visual embedding. ColiVara has state of the art retrieval performance on both text and visual documents. using vision models instead of chunking and text-processing for documents. No OCR, no text extraction, no broken tables, or missing images.
Things you can do with the token embeddings of an LLM
TokenHub gives enterprises a private gateway to unify AI model access and governance, making every request controllable, traceable, and attributable.
Non-destructive compression gateway for AI coding agents. Cuts token bills 25% on turn 1 to past 85% in long or saturated sessions, and fits ~3× more turns in the same context window. Powered by our open-source code-native 4B model. Drop-in for Claude Code, Cursor, Codex, OpenHands, and any BASE_URL agent.
Standalone CliProxyAPI usage tracker with SQLite persistence and built-in dashboard.
PDEBench: An Extensive Benchmark for Scientific Machine Learning
🚀 Convert Kiro accounts into OpenAI/Anthropic APIs. Features multi-account pooling, streaming, auto token refresh, and a web admin panel.
真实 API 定价:订阅月费 ÷ 实际可用 token,含额度、单价与分榜帕累托图。Real API pricing: subscription fee ÷ usable tokens, with allowance, unit-price, and leaderboard Pareto charts.
azooKey-Desktop is an open-source Japanese input method for macOS, written in Swift and powered by the Zenzai neural kana-kanji converter. It provides live conversion, optional LLM-based “Magic Conversions”, and Tuner-backed personalization for a smooth, desktop typing experience.