Step 18 of 22
Guardrails
Screen inputs and outputs for injection, leaks and policy breaks. Agents read untrusted text and take real actions. A cheap check in front of each step catches what the main model misses.
How to choose
- Prompt injection in fetched content: screen it before it enters context.
- Policy and topic rules: a rules-plus-model guardrail framework.
- Approving risky tool calls: a fast decision model or a human in the loop.
What to look for
The questions that separate the options, in order of how much they matter.
01
Which risks it actually detects
Map each option to the risks you face: prompt injection and jailbreaks, personal data and system-prompt leaks, toxic or off-policy content, off-topic requests and unsafe tool use. The OWASP Top 10 for LLM Applications is a useful checklist; LLM01 (prompt injection), LLM02 (sensitive information disclosure) and LLM06 (excessive agency) are the ones guardrails address most directly. A tool that only scores toxicity will not stop an injection hidden in a retrieved web page.
02
Where in the loop it can check
Input checks screen what the user sends; output checks screen the answer; for agents you also need checks on retrieved documents, tool inputs and tool results. NeMo Guardrails has separate rails for input, output, retrieval and tool execution, and Lakera Guard accepts any of them as separate requests. Indirect injection arrives through retrieval and tool output, so a guard that only sees the user message misses the most common agent attack.
03
False positives, measured on your own traffic
Every detector trades misses against wrongly blocked requests, and vendor accuracy figures are measured on their own test sets. Lakera documents four strictness levels and defaults to the strictest; Meta's Prompt Guard 2 card warns that accuracy drops on application-specific prompts. Before going live, run a few hundred real, benign requests and a set of known attacks through the guard, and record both the block rate and the miss rate at each threshold.
04
Latency and cost per check
Each model-based check adds a round trip before the user sees an answer, and an agent may run several per turn. Small classifiers are fast: Laya reports single-digit milliseconds on a GPU and GLiNER2.5-Decide a few hundred milliseconds on a CPU, while asking the main LLM to check itself can add seconds. Hosted APIs add network time and are usually billed per request, so count requests per agent turn, not per conversation.
05
Self-hosted or API
Guards see every prompt and response, so they handle your most sensitive data. Open frameworks and open-weight models (NeMo Guardrails, Guardrails AI, GLiNER2.5-Decide, Laya) run inside your infrastructure. Hosted services such as Jev or Lakera Guard are quicker to adopt but add a data processor; Lakera's self-hosted option needs an Enterprise licence.
06
How you define policy
Some options ship fixed detectors (injection, PII, moderation); others let you define your own labels or yes/no questions per request, which suits business rules such as 'is this a refund request over the limit'. Decision models like Jev and Laya are flexible but need you to write and test the questions. Frameworks such as NeMo Guardrails add flow logic in Colang, which is powerful but another language to learn.
07
Maintenance and ownership
This category is consolidating fast. Guardrails AI shut its hosted Hub in August 2026 and was acquired by Harvey; Protect AI's LLM Guard was archived in July 2026; Lakera now belongs to Check Point. Check the last release date and the owner's incentives, and keep the guard behind your own interface so it can be swapped.
How the options differ
Guardrails come in three families. Frameworks (NeMo Guardrails, Guardrails AI) wrap your LLM calls in a pipeline of checks and let you plug in detectors, from regex and PII tools to safety classifiers such as Llama Guard; they cover the most ground but need configuration. Hosted screening APIs (Lakera Guard) give you maintained detectors behind one endpoint, with tunable strictness and no models to run, at the cost of sending text to a vendor. Small classification and decision models (GLiNER2.5-Decide, Laya, Jev) answer a question you define, such as 'is this an injection attempt' or 'should this tool call be approved', with a probability you can threshold.
In practice teams combine them: a fast classifier or API on every input and retrieved document, a framework or decision model for business rules and tool approvals, and least-privilege tool permissions underneath, because no guard catches every adaptive attack.
6 tools, compared
Ranked by our rating. Open any review for sources, prices and the full verdict.
| Tool | Rating | Our verdict | Best for | Price |
|---|---|---|---|---|
| NVIDIA NeMo Guardrailsrepo | 4.0/5 | NeMo Guardrails is the broadest open policy layer for agents, and it charges for that breadth in configuration, latency and a changelog full of breaking changes. | Teams building agents or customer-facing assistants that need a configurable policy layer around every model and tool call. | Free and open source under Apache 2.0. NVIDIA's NemoGuard models and NIM microservices are distributed separately. |
| Lakera Guard (Check Point AI Guardrails)tool | 4.0/5 | Lakera Guard is a well-documented injection screen you can wire in with one API call, but past 10,000 requests a month you are negotiating with Check Point's enterprise sales team. | Teams that want a managed screen for prompt injection and data leaks in front of chatbots and agents, without operating their own classifiers. | Community free: 10,000 requests/month, 8k-token prompts, SaaS only. Enterprise by quote: flexible volume, SaaS or self-hosted, SSO, RBAC, SIEM integration. No published price beyond the free tier. |
| Jevmodel | 3.8/5 | Jev turned typed decisions into a product, and open models have since passed it on both score and price. | Teams that want a hosted, low-cost API for text routing, triage and yes/no gates without running their own GPUs. | $0.042 per 1M input tokens ($42 per billion); output tokens are free. Same price at long context. Enterprise limits through sales. |
| GLiNER2.5-Decidemodel | 3.6/5 | GLiNER2.5-Decide is a fast, honest label classifier that performs well on Fastino's tasks and far behind on everyone else's. | Engineers who need cheap, CPU-only intent routing, moderation or spam decisions and can fine-tune on their own labels. | Free weights and library under Apache 2.0. Fastino's hosted fine-tuning agent is a separate product. |
| Layamodel | 3.4/5 | Very fast and genuinely open, but close to guessing on hard decisions out of the box: a base to fine-tune, not a Jev replacement. | Teams with their own labelled decision data who want a tiny, self-hosted model to fine-tune for fast routing or guardrails. | Free weights and Python package under Apache 2.0. No hosted paid API; you run it yourself. |
| Guardrails AIrepo | 3.2/5 | A sound idea left without an owner: the Apache 2.0 code still runs, but nobody is merging fixes and the hosted pieces are gone. | Existing users with deterministic validators who can pin versions and patch it themselves. | Free and open source under Apache 2.0. The hosted Hub registry and free remote inference closed on 25 August 2026. |
Side by side
The specifications that matter for this step. Checked 2026-10-23; confirm on the vendor’s site before you commit.
| Tool | Type | What it checks | Deployment | Published latency | Licence |
|---|---|---|---|---|---|
| NVIDIA NeMo Guardrails | Rails framework (YAML + Colang) | Input, output, retrieval, tools, dialog | Self-host: library, server or NIM | Depends on rails used | Apache 2.0 |
| Lakera Guard (Check Point AI Guardrails) | Hosted screening API | Injection, PII, moderation, links, tools | SaaS; self-host on Enterprise | Under 50 ms (vendor claim) | Proprietary |
| Jev | Hosted decision model | Yes/no, choice or score you define | API only | 267–316 ms median | Proprietary |
| GLiNER2.5-Decide | Zero-shot classifier (under 0.5B) | Labels you define per request | Self-host, CPU or GPU | 294 ms on a desktop CPU | Apache 2.0 |
| Laya | Open decision model (421M) | Yes/no, choice or score you define | Self-host, CPU or GPU | 5.8 ms median | Apache 2.0 |
| Guardrails AI | Validator library | PII, toxicity, structure, custom | Self-host only (Hub closed) | Not published | Apache 2.0 |
Connect it to your agent
MCP servers that let assistants like Claude, ChatGPT and Cursor use these tools directly.
Where this stage fits
- Chat with your documents: we would start with NVIDIA NeMo Guardrails.
- Customer support agent: we would start with NVIDIA NeMo Guardrails.
- Coding agent: we would start with Jev MCP.
- Assistant with long-term memory: we would start with NVIDIA NeMo Guardrails.
- Image generation pipeline: we would start with GLiNER2.5-Decide.
- Video production: we would start with GLiNER2.5-Decide.
- Voice agent: we would start with NVIDIA NeMo Guardrails.
- Private, on-device AI: we would start with Laya MCP Server.
- Document processing: we would start with NVIDIA NeMo Guardrails.
- Your own fine-tuned model: we would start with NVIDIA NeMo Guardrails.
- Workflow automation: we would start with Jev MCP.