Adding guardrails to large language models.
Repository Index
77,852 visible repositories in the index
1–19 of 19 · Most stars
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
- agents
- guardrails
- python
- safety
Plano is an AI-native proxy server and data plane for agentic apps. Smart LLM routing, observability, agent orchestration, and guardrails so you stay focused on your agents core logic.
- gateway
- generative-ai
- llm-inference
- llms
Superagent protects your AI applications against prompt injections, data leaks, and harmful outputs. Embed safety directly into your app and prove compliance to your customers.
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
- ai-gateway
- observability
- simulation
- ai-agents
A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows
- agentic-ai
- agentic-workflow
- agents
- function-calling
A pre-execution guard for AI coding agents. It blocks destructive Git and file system commands, plus common attempts to access sensitive files, before a tool call runs. Supports Amp Code, Antigravity CLI, Claude Code, Codex, Cursor, DeepSeek Harness, Devin CLI, GitHub Copilot CLI, Grok Build, Hermes Agent, Kimi Code, OpenClaw, OpenCode, and Pi.
Local gateway for Claude Code, Codex and other AI clients on macOS, Windows and Linux: switch upstreams without touching clients, keep API keys from relays, cut off malicious tool calls, trace every request.
- ai-gateway
- anthropic
- claude-code
- codex
Local-first, open-source security gateway for AI coding agents. Enforce tool-call policies, detect secrets, and audit actions on-device.
- ai-agent-security
- ai-agents
- agent-security
- ai-security-tool
A local AI gateway for AI agents, unifying your subscriptions and API providers with smart routing and on-device privacy protection.
- ai-gateway
AI Agent Identity and Access Management and Inference Firewall
- agent-identity
- ai-audit
- ai-firewall
- ai-gateway
Open detection-rule standard for AI agent security threats — like Sigma, but for AI agents. Executable, testable rules for prompt injection, tool poisoning, context exfiltration and MCP attacks. Merged into open-source projects at Microsoft, Cisco, Gen Digital, MISP and FINOS. MIT-licensed.
- agent-security
- ai-security
- llm-security
- mcp-security
Find known secrets in your AI coding agent sessions (Claude Code, Codex, Cursor), then guard the next one with local hooks that trace risky commands to the content that dictated them. Open source, no telemetry.
- agent-security
- ai-agents
- ai-security
- claude-code
Runtime defense against memory poisoning in AI agents (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.
BLA blinds your agent. LASIK gives it its sight back. Static de-blinding CLI + MCP server that undoes BLA (Blind LLM Agents) hardening so coding agents and REA can read the real code again. Never executes the target.
- ai-agents
- anti-bla
- bla
- claude-code
[ACL 2025] The official implementation of the paper "PIGuard: Prompt Injection Guardrail via Mitigating Overdefense for Free".
Open standard for Provenance & Intent Contracts (PIC) in AI agents. Verify intent, provenance, and evidence before high-impact tool calls.
- action-proposals
- agentic-ai
- agents
- ai
Local-first policy gateway for coding agents. Allow, review, or block MCP, shell, SQL, GitHub, and HTTP actions with scoped credentials and signed evidence.
- golang
- observability
- opentelemetry
- policy-engine
Prompt-injection firewall for LLM applications — 33 input detectors, 9 output scanners, federated ed25519-signed threat-intel feed. Apache 2.0, 1040 tests, F1 96.0% with 0% false positives. Docker, GitHub Action, LangChain/LlamaIndex/CrewAI integrations.
- agentic-ai
- ai-safety
- fastapi
- langchain