Jailbreak (LLM)
A jailbreak is a prompt or technique that gets a language model to produce output its safety training is meant to refuse, such as harmful instructions, by role-play, obfuscation, odd encodings or long adversarial text.
Also called: jailbreaking, LLM jailbreak
By Divya Sharma · Checked Nov 15, 2026 · How we review
The word comes from removing restrictions on phones and game consoles. With LLMs nothing is installed or modified; the attacker only writes text. Common patterns include asking the model to play a character with no rules, splitting a forbidden request into harmless-looking parts, writing it in another language or encoding, or appending machine-generated strings that push the model past its refusal.
Why it works
Wei, Haghtalab and Steinhardt studied why safety training fails and proposed two causes. Competing objectives: the model is trained both to be helpful and to refuse, and a clever prompt sets those goals against each other. Mismatched generalisation: safety training does not cover every domain where the model has abilities, so a request phrased in an unusual form slips through.
Jailbreak vs prompt injection
People often mix the two. A jailbreak targets the model's own safety rules and is usually run by the user talking to it. Prompt injection targets the application's instructions, and is often planted by a third party in content the model reads. A single attack can be both: an injected web page might contain a jailbreak.
What builders do about it
Model providers patch known jailbreaks, but new ones keep appearing, so applications add their own guardrails such as input and output classifiers. Teams look for weak spots before release through red teaming, using tools like PyRIT or promptfoo. A jailbreak is a symptom of imperfect alignment: the model's behaviour does not fully match what its makers intended.
Related terms
- Prompt injection
- Prompt injection is an attack where text fed to a language model, typed by a user or hidden in a web page, file or tool result, changes the model's behaviour in ways its developer did not intend.
- AI red teaming
- AI red teaming is the practice of deliberately attacking a model or AI application before and after release, with human testers or automated tools, to find harmful outputs, jailbreaks, injections and other failures so they can be fixed.
- Guardrails (AI)
- Guardrails are runtime checks placed around a language model that inspect inputs and outputs and block, rewrite or redirect them, so an application stays on topic, avoids harmful content and follows rules the model alone would not reliably keep.
- AI alignment
- AI alignment is the work of making an AI system's behaviour match what its users and developers actually intend, so it is helpful, honest and avoids harm, rather than just predicting likely text or optimising a narrow score.
- Abliteration
- Abliteration is a technique that stops a language model from refusing requests by finding the single internal direction that carries refusal and editing the weights so the model can no longer write to it, without any retraining.