MobiInfer runs 35B-A3B-class MoE models fully on-device on the Qualcomm Hexagon NPU, streaming experts from flash with lookahead scheduling (LES) and DSpark speculation.
Repository Index
132,241 visible repositories in the index
697–720 of 748 · Most stars
Open source compiler for your whole ML repo (support python, CUDA C/C++) into sass and cuda binary (which are closed sourced by Nvidia); 3-40% faster than Openai Triton for Neural Network (primarily LLM) based training/inferencing
- compiler
- compiler-backend
- compiler-optimization
- gpu-acceleration
llama.cpp fork that abliterates at inference
Streaming-first load balancer for AI inference gateways
Deepinfer inspired implemetation on SM80 Ampere architecture. Deepest respect for DeepSeek opensource this work!
Translated🐋 Complete guide in Russian: how to install and run DeepSeek locally (V4, R1) on PC, Mac and server — Ollama, llama.cpp, LM Studio, vLLM, Open WebUI, API, RAG, agents, fine-tuning, best tips and tricks
- ai-agents
- deepseek
- deepseek-r1
- deepseek-v4
ModelSphere is an open-source LLM inference infrastructure designed to make production-grade model serving simple, efficient, and continuously optimized.
Feel how fast a local LLM runs before you buy the hardware
PROMETHEVS: run the full DeepSeek-V4.1-Flash-1017 (510 GB) on a gaming PC with 64 GB RAM and a 12 GB GPU. PROMETHEVS engine (compress-then-correct), OpenAI-compatible local server, report and logs.
A collection of awesome papers, articles and various resources on credit and credit risk modeling
- credit-scoring
- credit-modeling
- financial-machine-learning
- sample-selection
A Long-Horizon Benchmark for Self-Improving Inference Systems
inference and machine learning in clojure
Ruby FHIR Client
- fhir
- fhir-client
Production compatibility gateway bridge & observability platform connecting Claude Code CLI with Google Antigravity & Gemini inference models via Anthropic Messages API
PocketStrike is a privacy-first mobile AI workstation. It combines zero-cloud native on-device GGUF inference (ARM NEON accelerated) │ with frontier cloud LLMs, multi-step ReAct agent tools, Model Context Protocol (MCP) servers, self-evolving cross-session memory, and FLUX neural image │ generation.
This repository contains notebooks that show the usage of TensorFlow Lite for quantizing deep neural networks.
- tensorflow-2
- tensorflow-lite
- on-device-ml
- model-quantization
Probabilistic programming framework that facilitates objective model selection for time-varying parameter models.
- time-series-analysis
- model-selection
- bayesian-inference
- anomaly-detection
Source code of Local Code, a private AI agent for Mac, iPad and iPhone. Runs DeepSeek 4.1 on device, with Python, Node.js and a shell built into the app.
A inference framework that adapt to diverse and dynamic agent workload
- distributed-systems
- llm
- llm-agents
- system
Official implementation of "AReA: Adaptive Receptive-Field Attention for Efficient High-Resolution Diffusion Transformers"
- diffusion-models
- efficient-inference
- high-resolution
- image-generation
Solana DeFi Security: On-Chain Governance Risk Analysis for Multisig, Timelock, and Upgrade Authorit
- angulardart
- ark
- btc
- cam
Bringing thousands of learners together to unpack DeepInfer through peer-reviewed explanations, documentation, and hands-on examples.
Reverse Engineer any USB gadget: capture vendor-app traffic, infer the protocol with evidence, generate an open Python/libusb driver. CLI + MCP server.
InferPy: Deep Probabilistic Modeling with Tensorflow Made Easy
- probabilistic-programming
- neural-network
- tensorflow
- keras-tensorflow