What it is
NeMo Guardrails wraps an LLM application in rails: input rails check user messages, output rails check responses, retrieval rails check RAG content, execution rails check tool calls, and dialog rails steer conversations along flows written in Colang, NVIDIA's language for conversation logic (version 1.0 is the default; 2.0 is supported). Configuration is YAML plus Colang, extensible with Python.
Checks can use the main LLM to review itself, or dedicated models: NVIDIA's NemoGuard content-safety, topic-control and jailbreak-detection models, Meta's Llama Guard, and services such as Cisco AI Defense, ActiveFence, F5 Guardrails and Microsoft Presidio for PII. It runs as a Python library, as a server with an OpenAI-style endpoint, or as NVIDIA's Kubernetes microservice.
It is Apache 2.0-licensed. Version 0.24.0 (26 August 2026) extended the faster IORails engine to 59 of the library's 67 input and output checks, and 0.24.1 followed on 16 September 2026.
Our review
NeMo Guardrails is the broadest open toolkit for controlling an LLM app at runtime. It covers every point where things go wrong: what the user sends, what retrieval returns, what tools receive and what the model says back. Execution rails for tool calls make it more relevant to agents than most rivals, and development is brisk, with commits most days and three minor releases between July and September 2026.
It helps to be clear about what guardrails catch. Classifier checks such as Llama Guard 4 (a 12B model that labels text and images against 14 hazard categories) and Meta's Prompt Guard 2 (86M and 22M injection detectors) catch overt harmful content and known jailbreak patterns well. They miss subtler things. Meta's own card says Prompt Guard 2 can be beaten by adaptive attacks and loses accuracy on application-specific prompts, and Llama Guard's categories for defamation, IP and elections need facts it does not have. A determined indirect prompt injection hidden in a retrieved document can still get through. Guardrails reduce risk; least-privilege tool permissions remain the real defence.
Latency is the operational cost, and NVIDIA's answer is IORails, an engine that runs NemoGuard input and output checks in parallel in non-streaming mode, with admission control that returns an overload response rather than queueing forever. Since 0.24.0 it covers 59 of 67 checks, but configurations outside that set fall back silently to the older LLMRails engine unless you pass require_iorails=True, and parallel checking of streamed output still needs LLMRails. The 0.24.0 release alone listed six breaking changes, so pin versions.
One default surprises people. The library sends anonymous telemetry to NVIDIA when a rails object starts and then periodic heartbeats; NVIDIA says the payload holds versions and counts of configured rails, never prompts or keys, and DO_NOT_TRACK=1 or NEMO_GUARDRAILS_NO_USAGE_STATS=1 turns it off if set before startup. Guardrails AI is lighter for structured-output validation; this is closer to a policy engine, and it asks for the engineering time a policy engine deserves.
Where it earns its place
A platform engineer at a retail bank running a customer-facing assistant
Block jailbreaks and off-topic or unsafe replies before they reach customers
Run nemoguardrails server with input and output rails backed by NemoGuard content-safety and jailbreak models, using IORails with require_iorails=True so checks run in parallel. The library is free; the costs are a GPU for the classifiers and one added model round trip per rail.
An agent team giving an LLM write access to internal APIs
Check tool arguments and tool results before the agent acts on them
Execution rails inspect what each tool receives and returns, and retrieval rails can mask PII through Presidio before chunks reach the prompt. This narrows, but does not close, indirect injection risk; scoped API credentials still matter more.
A head of AI risk at an insurer who needs vendor support for audit
Put a supported guardrail service in front of several internal apps
Develop rails locally with the open library, then deploy the same YAML and Colang to NVIDIA's NeMo Guardrails microservice under NVIDIA AI Enterprise, listed at $4,500 per GPU per year for self-managed systems.
Before you commit
- Setup
- pip install nemoguardrails on Python 3.10 to 3.13, write a config folder of YAML and Colang, then nemoguardrails server --config ./config --port 8000 for an OpenAI-style /v1/chat/completions endpoint.
- Integrations
- LangChain wrapper; NemoGuard, Llama Guard, Cisco AI Defense, ActiveFence, F5 Guardrails, Fiddler, Cleanlab and Presidio; OpenTelemetry tracing to your own backend; Helm-deployed microservice for Kubernetes.
- Data and privacy
- Telemetry is on by default and also written to ~/.config/nemoguardrails/usage_stats.json; opt out with DO_NOT_TRACK=1. Rails that call hosted models send text to those providers, and NVIDIA says its build.nvidia.com endpoints are for evaluation only, not production or personal data. No advisories for the package appear in the GitHub advisory database.
- Cost at scale
- The library is free under Apache 2.0. Real cost is extra inference: every model-based rail adds a call per message, self-check rails use your main LLM, and NemoGuard classifiers need GPU capacity. NVIDIA AI Enterprise, which covers the microservice, lists at $4,500 per GPU per year.
- Limits
- About 270 open issues and pull requests; IORails supports 59 of 67 checks and no parallel streaming output rails; minor releases carry breaking changes; classifiers miss adaptive and indirect injections.
What's good
- Covers input, output, retrieval, tool and dialog checks
- Plugs in Llama Guard, NemoGuard, Presidio and commercial detectors
- IORails runs NemoGuard checks in parallel to cut added latency
- Library, OpenAI-style server or Kubernetes microservice
What's not
- Anonymous telemetry to NVIDIA is on by default
- Breaking changes in minor releases
- Colang adds a learning curve
- Best detectors steer you towards NVIDIA models
Best for
Teams building agents or customer-facing assistants that need a configurable policy layer around every model and tool call.
Alternatives
NVIDIA NeMo Guardrails: common questions
- Is NVIDIA NeMo Guardrails worth it?
- NeMo Guardrails is the broadest open policy layer for agents, and it charges for that breadth in configuration, latency and a changelog full of breaking changes. We rate it 4.0 out of 5.
- How much does NVIDIA NeMo Guardrails cost?
- Free and open source under Apache 2.0. NVIDIA's NemoGuard models and NIM microservices are distributed separately.
- What are the downsides of NVIDIA NeMo Guardrails?
- Anonymous telemetry to NVIDIA is on by default; breaking changes in minor releases; colang adds a learning curve; best detectors steer you towards NVIDIA models.
- Who is NVIDIA NeMo Guardrails best for?
- Teams building agents or customer-facing assistants that need a configurable policy layer around every model and tool call.
- How much does NVIDIA NeMo Guardrails cost for a team or in production?
- The library is free under Apache 2.0. Real cost is extra inference: every model-based rail adds a call per message, self-check rails use your main LLM, and NemoGuard classifiers need GPU capacity. NVIDIA AI Enterprise, which covers the microservice, lists at $4,500 per GPU per year.
- Is NVIDIA NeMo Guardrails safe for company data?
- Telemetry is on by default and also written to ~/.config/nemoguardrails/usage_stats.json; opt out with DO_NOT_TRACK=1. Rails that call hosted models send text to those providers, and NVIDIA says its build.nvidia.com endpoints are for evaluation only, not production or personal data. No advisories for the package appear in the GitHub advisory database.
- What are the best NVIDIA NeMo Guardrails alternatives?
- The closest alternatives we have reviewed are Guardrails AI, vLLM Semantic Router and promptfoo.
Answers checked Oct 23, 2026.
Make NVIDIA NeMo Guardrails? Show this rating on your site
