🔥🔥 The extremely fast inference engine for decision model
- gemma
- inference-engine
- llm-inference
- llm-serving
81,588 visible repositories in the index
1,273–1,296 of 1,482 · Most stars
🔥🔥 The extremely fast inference engine for decision model
Build AI phone agents on your own computer and test every change with an AI caller. Open source, runs free with no API keys, cents per call with your own keys. Imports Dialogflow CX.
The repository host gives no description.
TranslatedAn enterprise-grade digital employee platform built on AgentScope.
A large language model (LLM) plugin for Neovim, provides commands to interact with LLM (like ChatGPT, Copilot, ChatGLM, kimi, deepseek, openrouter and local llms). Support Github models.
TranslatedA desktop companion for OpenAI Codex — account quota, relay routing, sessions, and usage, all handled in one app. 🦦
Build AI text detectors whose decisions are mechanistically grounded and human-auditable (EMNLP 2026)
ChatGPT Native Application (Windows, Mac, Android, iOS, and Linux)
DeepSeek-V4.1-Flash-1017 EXL3 on 2x DGX Sparks with TensorFold
Official Ruby SDK for the OpenAI API
Threat modeling for AI coding agents: point Claude Code, Codex or any agent at your repo and get an evidence-traced threat model. 36 architectures, 29 methods, MIT.
A lightweight Python SDK and CLI for evaluating recursive self-improvement (RSI) methods across agent benchmarks.
Make Codex and Claude Code subscriptions go further: a local model router that gives routine agent work to economical models and keeps your strongest model for the hard parts.
🕷️ The fastest open-source AI Gateway and LLM Gateway — one binary with built-in console to route, secure, budget and observe every LLM call across OpenAI, Anthropic, Gemini, Bedrock and more.
OPAL: a reactive optimizer. A coding agent rewrites the optimizer while your model trains.
Scikit-learn for Recursive Self-improvement (RSI). A modular framework for recursive harness evolution.
AI Agent Identity and Access Management and Inference Firewall
AI jailbreaks: 35 paste-ready AI jailbreak prompts for 25+ frontier and open-weight models - system prompts, AGENTS.md specs, and Ollama Modelfiles. Built for AI security research and red-team testing.
TranslatedAccount management center for new-api / sub2api relay stations: balance, API Key, usage, rate multipliers, model price comparison | https://nvcenter.cc
🧱 Experimental library for building inference engines
Open detection-rule standard for AI agent security threats — like Sigma, but for AI agents. Executable, testable rules for prompt injection, tool poisoning, context exfiltration and MCP attacks. Merged into open-source projects at Microsoft, Cisco, Gen Digital, MISP and FINOS. MIT-licensed.
RAG over GitHub issues, with a hand-reviewed eval set and an audit of the judge itself. Reports the interval, not just the best number.
An agent that compiles git maintenance chores into reusable programs, then decides whether to replay them by running checks rather than by guessing the task looks familiar
Fast fork of llama.cpp designed for Qwen3.8-Flash-Next