DeepInfer: High-performance, large-scale inference framework for DeepSeek models
Repository Index
47,133 repositories indexed
1–24 of 84 · Most stars
Jev-like family of decision models built on top of Qwen3.5/3.8 you can train and run on your own
- decision-model
- jev
- qwen3
Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported
TranslatedOpen-source manuscript of 《深入理解 AI Infra：量化分析与系统设计》 (In-Depth Understanding of AI Infra: Quantitative Analysis and System Design), by 李博杰 (Li Bojie): starting from hardware constraints and model architecture, it quantitatively derives the system design of LLM inference and training. Includes the full text of the book, a PDF, and companion calculation tools and experiments
- accelerator
- ai-infra
- ai-infrastructure
- book
TranslatedWorkBuddy international and domestic multi-account reverse proxy gateway, supporting Codex / Claude Code / DSH and standard OpenAI clients.
- api-gateway
- claude-code
- codebuddy
- codex
An open-source document parsing model with SOTA accuracy and speed, using an RSI loop during post-training.
Infrastructure for the next generation of voice agents, designed to provide universal memory. It is divided into a left brain and a right brain, storing information and emotions respectively, while a fully streaming architecture eliminates latency at the fundamental level.
Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.
- kv-cache
- llm-inference
- local-llm
- quantization
TranslatedTencent CodeBuddy account pool management console with a built-in OpenAI-compatible reverse proxy gateway: bulk onboarding of accounts via QR code scan, scheduled check-ins and keep-alive, key grouping and distribution, IP and model whitelists, call logs and usage statistics; supports one-click updates with signature verification.
- account-manager
- api-gateway
- codebuddy
- fastapi
Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results
- interpretability
- mechanistic-interpretability
- qwen
- steering
Open decision models from 0.8B to 27B: a calibrated probability for every option in one pass, and a short thought only when unsure. Engine for vLLM, llama.cpp and MLX.
- agents
- calibration
- classification
- decision-making
1M param OCR model - runs on CPU, 15x faster than tesseract, close to 1B+ param models in accuracy
Turn any LLM into a Jev-style decision model: typed decisions, real probabilities, no training. (continue updating, welcome any issue and PR request)
- calibration
- decision-model
- jev
- jev-model
A free AI assistant and survival library that runs on your own computer, with no internet needed. Shows the source of every answer.
Trace-native CI/CD for AI agents: grade production traces, cluster failures, freeze them into replayable regression tests, and block PRs that regress. Self-hostable.
GVS5H: Five Qwen3.8-27B Models Match Claude Fable 5 on LiveCodeBench Hard | Fable 5 Level Coding for a Fifth the Price - or on a Single GPU
Official Inference Code of EAGLE-4
- eagle
- gpu
- llm-inference
- sglang
A clinical decision-evidence benchmark: pull the one fact an answer rests on, and see whether the assistant's action moves with the evidence. Built on HealthBench.
TranslatedDeepSeek Harness integrated armor-breaking security analysis plugin: aggregating all domains of Android · Web · Native · Protocol · Malware · AI-Security (9 bundles + 1 preset)
EdgeRock: Strong Reasoning at Constant Memory. Multimodal models with O(N) generation and O(1) persistent history. Inference and evaluation code.
Native vLLM serving for Jev decision models
TranslatedOpenAI-compatible API for Muse.ai with Chat, Image & Video generation. It wraps Muse (muse.ai) via reverse engineering as an OpenAI-compatible interface, supporting chat, text-to-image, text-to-video/image-to-video, multi-account pool rotation, and 48h automatic renewal.
TranslatedAI Cross-Border E-commerce Operations Agent Toolkit · Breaks operations SOPs down into 7 reusable AI Agent workflows, including prompts, templates, and real-world cases
FactCircuit: an auditable claim-and-evidence verification loop with immutable provenance, staged checks, bounded retrieval, and reproducible evaluation.
- accuracy-decline
- ai-safety
- audit-trail
- fact-checking