What is LlamaFirewall?
LlamaFirewall is a Python framework from Meta that runs security scanners at chosen points in a chatbot or agent loop: on user input, on untrusted content such as web pages and tool output, on the agent's reasoning and on code it writes. It lives in the PurpleLlama repository alongside Meta's Llama Guard models and the CyberSecEval benchmarks.
It ships four kinds of scanner. PromptGuard 2 is a small BERT-style classifier for direct prompt injection and jailbreaks. AlignmentCheck asks a large model to read the agent's chain of thought and flag goal hijacking or indirect injection. CodeShield is a static analyser for generated code, using Semgrep and regex rules across eight languages. Regex and custom LLM-prompt scanners cover secrets and house rules.
It installs with pip install llamafirewall on Python 3.10 or later, downloads its guard models from Hugging Face (Llama model access is required) and has examples for the OpenAI Agents SDK and LangChain. AlignmentCheck calls a hosted model through Together AI by default, so it needs a Together API key. The code is MIT-licensed; the last PyPI release, 1.0.3, came out on 29 May 2025.
Is LlamaFirewall worth it?
LlamaFirewall's value is the evidence that came with it. Meta's paper measured each layer on AgentDojo, a benchmark of agents doing tasks while attackers plant instructions in their tools. With no guardrail, 17.6% of attacks succeeded. PromptGuard 2 86M alone cut that to 7.5% with almost no loss of task completion; AlignmentCheck running on Llama 4 Maverick cut it to 2.89%; both together reached 1.75%, a reduction of more than 90%, while task success fell from 47.7% to 42.7%.
That trade-off is the useful lesson. The cheap classifier is fast, 19.3 ms for the 22M model and 92.4 ms for the 86M on an A100 at 512 tokens, but it only sees text that looks like an injection. AlignmentCheck catches subtler hijacks because it reads what the agent is trying to do, and pays for it: the paper's example trace shows roughly 860 ms and 1,490 ms per step, and it sends the agent's reasoning to whichever model provider you configure.
CodeShield is the quiet strength. Meta reports 96% precision and 79% recall on hand-labelled completions, with about 90% of inputs cleared in a first pass of around 60 ms. For a coding agent that writes and runs code, that is a cheap check on more than 50 weakness classes.
The catch: this is a research release in maintenance. The PyPI package is still 1.0.3 from May 2025, and recent commits to the folder are automated type-checker and dependency fixes. The PurpleLlama top-level README does not describe it at all. The scanners are sound; expect to own the glue code and upgrades yourself.
Our take: LlamaFirewall reads best as a reference design. Its PromptGuard 2 and CodeShield layers are worth running as they are, and the AlignmentCheck idea is worth copying with whichever large model you already pay for.
Where it earns its place
A platform engineer running a browsing agent built on the OpenAI Agents SDK
Stop instructions hidden in fetched pages from redirecting the agent
Wire PromptGuard 2 on every tool result and AlignmentCheck on each step, starting from Meta's OpenAI Agents demo. Expect the classifier to add tens of milliseconds and the alignment audit around a second per step, plus Together AI token costs.
A developer-tools team whose coding agent executes what it writes
Catch insecure code before the agent runs or commits it
Run CodeShield on each generated snippet. Most inputs clear the first pass in about 60 ms; flagged code goes to a slower second tier or to a person.
A security architect at a bank drafting an agent guardrail standard
Justify which checks go where, with numbers
Cite the paper's AgentDojo results, 17.6% attack success with no guard against 1.75% with both layers, and the 5-point drop in task success, to set the accepted cost of each layer.
Before you commit
- Setup
- pip install llamafirewall on Python 3.10+, accept Meta's Llama licence on Hugging Face, run llamafirewall configure to fetch the guard models, and export TOGETHER_API_KEY for AlignmentCheck.
- Integrations
- Python API with role-based scanner policies; examples for the OpenAI Agents SDK and LangChain or LangGraph agents; custom regex and LLM-prompt scanners.
- Data and privacy
- PromptGuard 2, CodeShield and regex scanners run on your machine. AlignmentCheck sends the conversation and agent reasoning to the configured model provider, Together AI by default.
- Cost at scale
- The framework is free. Classifier and CodeShield checks cost local CPU or GPU time; AlignmentCheck costs one large-model call per audited step, billed by your provider.
- Limits
- PromptGuard 2 reads 512 tokens at a time; AlignmentCheck is labelled experimental in the paper; last package release May 2025, with recent commits limited to automated maintenance.
What’s good
- Published AgentDojo numbers for each layer and for the combination
- Scanners for input, tool output, agent reasoning and generated code
- CodeShield: 96% precision on Meta's labelled set, about 60 ms first pass
- MIT licence; OpenAI Agents SDK and LangChain examples
What’s not
- No PyPI release since 1.0.3 in May 2025
- AlignmentCheck needs a hosted large model and a Together API key by default
- AlignmentCheck adds around a second per agent step
- Not documented in the PurpleLlama top-level README
Best for
Teams building tool-using agents who want a layered, self-hostable guard and can maintain an unpolished framework.
LlamaFirewall alternatives
LlamaFirewall: common questions
- Is LlamaFirewall worth it?
- The paper behind it makes the best public case for layered agent guardrails, but the package itself has not had a release since May 2025 and its strongest check depends on a large hosted model. We rate it 3.8 out of 5.
- How much does LlamaFirewall cost?
- Free under the MIT licence. PromptGuard 2 and CodeShield run locally; AlignmentCheck calls a hosted model through Together AI by default, billed by Together.
- What are the downsides of LlamaFirewall?
- No PyPI release since 1.0.3 in May 2025; alignmentCheck needs a hosted large model and a Together API key by default; alignmentCheck adds around a second per agent step; not documented in the PurpleLlama top-level README.
- Who is LlamaFirewall best for?
- Teams building tool-using agents who want a layered, self-hostable guard and can maintain an unpolished framework.
- How much does LlamaFirewall cost for a team or in production?
- The framework is free. Classifier and CodeShield checks cost local CPU or GPU time; AlignmentCheck costs one large-model call per audited step, billed by your provider.
- Is LlamaFirewall safe for company data?
- PromptGuard 2, CodeShield and regex scanners run on your machine. AlignmentCheck sends the conversation and agent reasoning to the configured model provider, Together AI by default.
- What are the best LlamaFirewall alternatives?
- The closest alternatives we have reviewed are NVIDIA NeMo Guardrails and Guardrails AI.
Answers checked Nov 6, 2026.