LLM inference in C/C++
- ggml
106,102 visible repositories in the index
1–24 of 207 · Most stars
LLM inference in C/C++
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
Protocol Buffers - Google's data interchange format
Port of OpenAI's Whisper model in C/C++
Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.
Official inference framework for 1-bit LLMs
Cross-platform, customizable ML solutions for live and streaming media.
FinceptTerminal is a modern finance application offering advanced market analytics, investment research, and economic data tools, designed for interactive exploration and data-driven decision-making in a user-friendly environment.
SRS is a simple, high-performance, AI-driven real-time media server supporting RTMP, WebRTC, HLS, HTTP-FLV, HTTP-TS, SRT, MPEG-DASH, and GB28181, with codec support for H.264, H.265, AV1, VP9, AAC, Opus, and G.711.
Productive, portable, and performant GPU programming in Python.
Distribute and run LLMs with a single file.
FlatBuffers: Memory Efficient Serialization Library
ncnn is a high-performance neural network inference framework optimized for the mobile platform
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Offline documentation browser. Your personal reference library, searchable in an instant.
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
Krita is a free and open source cross-platform application that offers an end-to-end solution for creating digital art files from scratch built on the KDE and Qt frameworks.
High-performance GPGPU inference of OpenAI's Whisper automatic speech recognition (ASR) model
A high-performance distributed file system designed to address the challenges of AI training and inference workloads.
Production ready toolkit to run AI locally
High-speed Large Language Model Serving for Local Deployment
WinMerge is an Open Source differencing and merging tool for Windows. WinMerge can compare both folders and files, presenting differences in a visual text format that is easy to understand and handle.
Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.