LLM inference in C/C++
- ggml
Repository Index
65,264 visible repositories in the index
1–24 of 68 · Most stars
GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
- llm-inference
- ai-chat
Welcome to a calmer internet
- firefox
- firefox-based
- firefox-browser
- zen-browser
Official inference framework for 1-bit LLMs
Distribute and run LLMs with a single file.
- cross-platform
- gguf
- llama-cpp
- local-ai
WebRTC/RTSP/RTMP/HTTP/HLS/HTTP-FLV/WebSocket-FLV/HTTP-TS/HTTP-fMP4/WebSocket-TS/WebSocket-fMP4/GB28181/SRT/STUN/TURN server and client framework based on C++11
MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.
- mnn
- convolution
- vulkan
- embedded-devices
A lightweight, lightning-fast, in-process vector database
- rag
- agent-skills
- embedded
- faiss
COLMAP - Structure-from-Motion and Multi-View Stereo
- structure-from-motion
- multi-view-stereo
- reconstruction
- geometry
A suite of WiFi/Bluetooth offensive and defensive tools for the ESP32
KoboldCpp - Run GGUF AI models locally with a KoboldAI UI. One File. Zero Install.
OpenGL Mathematics (GLM)
- glm
- opengl
- mathematics
- vector
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
- inference
- deep-learning
- openvino
- ai
Cross-platform music production software
Production ready toolkit to run AI locally
Real-Time SLAM for Monocular, Stereo and RGB-D Cameras, with Loop Detection and Relocalization Capabilities
High-speed Large Language Model Serving for Local Deployment
Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.
Synchronous multiroom audio player
- audio-player
- sonos
- lms
- synchronous
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.
- edge-ai
- on-device-ai
- on-device-llm
Dusklight brings a classic adventure to PC and mobile platforms with a variety of fixes and improvements.
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
- amd
- llama
- llm
- llm-inference
2bit quants, kernels and runtime for consumer devices.