FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B
- cuda
- cuda-kernels
- realtime-inference
- realtime-vla
Video Frame Interpolation & Super Resolution using NVIDIA's TensorRT & Tencent's NCNN inference, beautifully crafted and packaged into a single app
TensorRT C++ API Tutorial
Frontier-class open models on a free Kaggle TPU v5e-8: GLM-5.3-Flash 320B MoE (~165 tok/s, our own JAX engine) and Qwen3.8-27B bf16 (~130 tok/s), 262k context, prefix caching. Works with Claude Code, Codex, opencode and pi.
An open-source solution for full parameter fine-tuning of DeepSeek-V3/R1 671B, including complete code and scripts from training to inference, as well as some practical experiences and conclusions. (DeepSeek-V3/R1 满血版 671B 全参数微调的开源解决方案,包含从训练到推理的完整代码和脚本,以及实践中积累一些经验和结论。)
PhoBERT: Pre-trained language models for Vietnamese (EMNLP-2020 Findings)
- python3
- bert
- bert-embeddings
- pos-tagging
Statistical Inference via Data Science: A ModernDive into R and the Tidyverse
Hands-on LLM algorithms and systems with PyTorch, Triton & CUDA.《大模型算法与系统教程》:模型架构与训练、后训练、推理加速、性能与算子优化,配套 Notebook 练习、答案解析、测试与实验。
Adlik: Toolkit for Accelerating Deep Learning Inference
Gateway API Inference Extension
Free AI coding environment: Ollama + Python
- coding-assistant
- local-inference
- ollama
⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
DoubleML - Double Machine Learning in Python
Inference repo for Falcon-Perception and Falcon-OCR model, early-fusion, natively multimodal, dense Autoregressive Transformer models.
Run TensorFlow models in C++ without installation and without Bazel
An R package for tidyverse-friendly statistical inference
Translated斑海豹 (Spotted Seal) — Phocinae-Largha-150M-v1: a 144.3M bilingual decision model. Tiny model. Big decisions.
- bilingual
- calibration
- cpu-inference
- decision-making
A Python library for simulation-based inference with deep learning
- deep-learning
- amortized-inference
- bayesian-statistics
- computational-modeling
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
- local-ai
- ai-sdk
- decentralized-ai
- llama
AI Inferencing at the Edge. A simple one-file way to run various GGML models with KoboldAI's UI with AMD ROCm offloading
Vui Nano — a small, context-aware text-to-speech model trained on real conversations. 219M active params (305M total), Apache 2.0, voice cloning, streaming, runs on CPU (dependency-free C build). Ships with a full real-time voice assistant: WebRTC, ASR, local LLM, OpenAI Realtime API compatible.
- conversational-ai
- edge-ai
- lightweight
- multi-speaker
The balance python package offers a simple workflow and methods for dealing with biased data samples when looking to infer from them to some target population of interest.
Small, dependency-free, fast Python package to infer binary file types checking the magic numbers signature
- magic-numbers
- filetype
- python
- mime
Machine Learning inference engine for Microcontrollers and Embedded devices