In an AI context
Model providers train refusal behaviour into their models: a set of outputs the model is intended to decline regardless of how it is asked. A jailbreak is any technique that gets past it.
The techniques change constantly and the categories are stable. Role-play framings that place the request inside a fiction. Hypothetical or academic framings that recast a prohibited request as analysis. Incremental escalation across a long conversation, where no single turn is refusable. Encoding, translation or obfuscation that moves the request outside the distribution the refusal training covered. Persona instructions that assert an alternative operating mode.
What they share is that they exploit the gap between what refusal training generalised to and what it was tested against. That gap is a property of the model, which is why a jailbreak that works against a model in one product usually works against it in another — and why it stops working when the provider ships an update, and often starts working again in a slightly different form shortly after.
How this differs from prompt injection
The distinction is about what is being attacked, and it determines who can fix it.
A jailbreak attacks the model's alignment. It is a claim about the model: this model, asked this way, will produce output it was trained to refuse. The behaviour is portable across applications and the primary remedy — better refusal training — belongs to whoever trained the model.
Prompt injection attacks an application's context assembly. It is a claim about your system: this application places untrusted content in the same context as its instructions, so content can instruct it. It is not portable, because it depends on how you built the thing, and the remedy is yours.
The confusion is understandable, because indirect prompt injection is often how a jailbreak reaches a model that a user could not address directly. But the governance consequence of confusing them is expensive: a firm that classifies its exposure as "jailbreak risk" concludes it is waiting on its vendor, when the exploitable surface is its own tool authorisation.
A rough test: if the same trick would work against the same model inside a competitor's product, it is a jailbreak. If it depends on what your application feeds the model, it is injection.
Why a refusal is not a control
The most common evidence offered that a system is safe is a transcript of it refusing. This is weak evidence and it is worth being explicit about why.
Refusal is probabilistic, version-dependent and untestable at the boundary. The same request phrased slightly differently may not be refused. A model update can change refusal behaviour in either direction without notice. And there is no way to enumerate the space of phrasings that would succeed, so no test suite can establish coverage — only the absence of the failures you thought to try.
Refusal behaviour is a useful defence in depth and a poor control to rely on. Where a specific output would be genuinely harmful, the reliable answer is not to trust the model to decline: it is to constrain what happens downstream, filter output against the specific prohibition, and restrict what the system can act on. A control you can test deterministically beats one you can only sample.
What a deploying organisation can actually do
You did not train the model, so alignment is not your lever. Four things are.
Choose and monitor the provider. Refusal robustness varies between models and between versions of the same model. It is a legitimate procurement criterion, and it belongs in periodic supplier review rather than only at selection — provider behaviour changes under you.
Filter output against your own prohibitions, which are usually narrower and more concrete than the provider's general safety policy. A firm knows what it must never say to a customer far more precisely than a general-purpose model does.
Reduce the consequence. As with injection: if a jailbroken response can only reach a human reviewer, the exposure is bounded. If it can reach a customer, be published, or trigger an action, it is not.
Log and review, because you will not prevent every case. A jailbreak you never detected is one you cannot report, learn from, or evidence you handled. Detection matters more than an unattainable prevention rate.
Where it appears in frameworks
OWASP treats jailbreaking within LLM01: Prompt Injection, which is a defensible taxonomy choice for a threat list and unhelpful for a governance programme — the two have different owners and different controls. This entry keeps them separate for that reason.
MITRE ATLAS catalogues jailbreak techniques as adversary behaviours, and is the better reference for modelling a specific attack.
The EU AI Act brings jailbreak resistance inside the robustness and cybersecurity requirements for high-risk systems, and — separately — makes it a transparency question: a system induced to deny it is an AI system engages Article 50(1) whatever else it does.
NIST's Generative AI Profile treats the harms a jailbroken model can produce as a risk category in their own right, independently of the mechanism.
Frequently asked questions
What is an LLM jailbreak? A technique that gets a language model to produce output its safety training was meant to refuse — typically by reframing the request as fiction, hypothesis or analysis, escalating gradually, or obfuscating it beyond what the refusal training covered.
Is jailbreaking the same as prompt injection? No. A jailbreak attacks the model's safety training and would work against that model in anyone's product. Prompt injection attacks how your application assembles untrusted content into the model's context. Injection is often how a jailbreak is delivered, which is why the terms get conflated — but the fixes belong to different parties.
Can jailbreaks be prevented? Not reliably, by anyone. Refusal training is probabilistic and changes between model versions, and the space of possible phrasings cannot be enumerated. The achievable goal is to reduce the consequence and detect occurrences, not to reach a prevention rate.
Is jailbreaking illegal? Testing a model you are authorised to use, against your own systems, is ordinary security work. Doing it to someone else's production system without authorisation is not, and using a jailbroken model to produce genuinely harmful content carries its own consequences independent of the technique. As with any security testing, authorisation and scope are what separate the two.
What should we do if our system is jailbroken? Treat it as a security incident: contain it, work out what the model could actually reach and whether anything downstream acted on the output, and check whether the same path allowed indirect prompt injection — the two frequently share a route in.