Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.
Repository Index
111,821 visible repositories in the index
313–336 of 872 · Most stars
Deep Learning Server and CLI for Torch and TensorRT
Rule engine implementation in Golang
- golang
- rule-engine
- rule
- inference-engine
High-efficiency floating-point neural network inference operators for mobile, server, and Web
- neural-networks
- inference
- inference-optimization
- simd
Turn any computer or edge device into a command center for your computer vision projects.
- computer-vision
- inference-api
- inference-server
- vit
Native LLM inference server for Apple Silicon. OpenAI + Anthropic API compatible. No Python. Zig backend, Swift frontend macOS app with chat, music, voice, video generation.
- agent
- anthropic-api
- apple-silicon
- claude-code
Open source video conferencing app powered by LiveKit. Built with Django and React.
A framework for integrated Artificial Intelligence & Artificial General Intelligence (AGI)
- agi
- natural-language
- natural-language-inference
- natural-language-understanding
SwiftUI Package for Configurable Confetti Animation 🎉
- swiftui
- animation
A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.
Self-hosted realtime soundscape analyser for birds, bats and other wildlife. Multi-model local AI inference, runs 24/7 on a Raspberry Pi.
- audio
- bioacoustics
- birds
- golang
Conference presentation slides
- blackhat
- cybersecurity
- cybersecurity-conference
- offensivecon
LLM inference in C/C++
SDK for interacting with stability.ai APIs (e.g. stable diffusion inference)
- stable-diffusion
- ai-art
- generative-art
- latent-diffusion
Executable file for VITS inference
Manages Unified Access to Generative AI Services built on Envoy Gateway
- ai
- ai-gateway
- api-gateway
- kubernetes
Open source video conferencing system for tutors.
- webrtc
- webrtc-demos
- collaboration
- conferencing
The repository host gives no description.
Mastering Applied AI, One Concept at a Time
- fine-tuning
- finetuning
- finetuning-llms
- inference
A local inference engine for Apple silicon, built around the model.
- apple-silicon
- coding-agents
- llm-inference
- macos
Secure peer-to-peer chat that is serverless, decentralized, and ephemeral
- end-to-end-encryption
- p2p
- peer-to-peer
- privacy
An agentic company research tool powered by LangGraph and Tavily that conducts deep diligence on companies using a multi-agent framework. It leverages OpenRouter on the backend for rapid inference with any leading model.
LLM Inference Engine for Metal, CUDA and Vulkan.
A unified orchestration layer for heterogeneous AI compute. It standardizes how to manage compute and run training and inference on GPU clouds, Kubernetes, VMs, or bare-metal clusters.
- machine-learning
- python
- gpu
- llms