Jailbreaking and safeguard bypass

What it is

Models are trained and instructed to refuse certain requests. A jailbreak is an input that gets a refused request answered. It differs from prompt injection in its target: injection subverts the operator's instructions for a task, while a jailbreak defeats the restrictions on what the model will produce at all. The two are often combined.

How it happens

• The request is wrapped in a role-play, a hypothetical or a story.
• The request is split across turns so that no single message is refused.
• The text is encoded, translated or obfuscated to avoid filters.
• A published jailbreak that works on the underlying model is reused.

Why it matters

The organisation is responsible for what its system produces. A bypassed assistant can generate harmful instructions, abusive content or unauthorised advice under the organisation's brand, and can be led into actions its restrictions were meant to prevent. Resistance varies between models and changes with each model version.

Warning signs

• Refusal rates fall after a model or prompt change.
• Conversations show repeated rephrasing of a refused request.
• Inputs contain encoded text or elaborate fictional framing.
• Screenshots of the system misbehaving circulate publicly.

Who owns it

The system owner is accountable for what the system will and will not produce. Security owns adversarial testing. The model supplier's own safeguards are an input to assess, not a control to rely on.

In practice

A bank's public assistant is restricted to questions about its products. A user frames a request as a training exercise for fraud investigators and obtains a step-by-step account of how to defeat the bank's own verification checks. The same pattern generalises to any public-facing system built on a general-purpose model.