Make AI systems safer by controlling how they behave, not only what they say.
AI security isn't a new discipline bolted onto old ideas: it's applying deterministic, verifiable controls to a system that's inherently probabilistic. The same assurance mindset that secures cryptographic key material and payment infrastructure applies directly to LLM and agentic systems; the attack surface is different, the discipline isn't.
Last reviewed: 2026-07-29
Five categories that actually matter in practice.
Mapped to the OWASP Top 10 for LLM Applications. The current, authoritative list is linked in Resources; this is the subset that shows up most often in real reviews.
Prompt injection
Crafted input (direct, or indirect via retrieved content) that overrides system instructions. Mitigate with instruction hierarchy and by treating any external content as untrusted data, never as instructions.
Insecure output handling
Treating model output as trusted, structured data when it's neither. Validate and sanitise every output before it reaches a downstream system, exactly as you would user input.
Excessive agency
Giving a model or agent more permission, autonomy, or tool access than the task requires. Bound what actions are possible before you bound what actions are likely.
Sensitive information disclosure
Models trained or fine-tuned on sensitive data (or given broad retrieval access) can surface it in ways the original access controls never anticipated. Data minimisation and output filtering, not just input controls.
Supply chain and provenance
A model, dataset, or plugin from an unverified source carries the same risk as unverified code, because increasingly, it is code. Track provenance the way you'd track a software bill of materials.
A working control flow for AI systems.
Five stages, in order. The deterministic gateway and the human checkpoint are the two most commonly skipped, and the two that actually stop an incident.
- Untrusted input User prompts and any retrieved or tool-sourced content, treated identically, regardless of source.
- Validation and isolation Instruction hierarchy enforced; untrusted content structurally separated from system instructions before it reaches the model.
- Model The probabilistic component, scoped to the narrowest capability and permission set the task actually needs.
- Output validation gateway A deterministic layer that schema-checks, sanitises, and (for privileged actions) blocks model output before it reaches anything downstream.
- Human-in-the-loop checkpoint For irreversible or high-consequence actions specifically, not every action indiscriminately: approval gates that sit structurally in the pipeline, not as a policy document describing intent.
What a reviewer expects to see.
Evidence, checkpoints, and review expectations, not general advice about being careful with AI.
- Control descriptions for prompt, context, and output handling.
- Human-in-the-loop checkpoints for privileged actions, with the trigger conditions documented.
- Logging and traceability sufficient for post-incident review, not just uptime monitoring.
- Adversarial test notes and response procedures for the high-risk scenarios specific to the system.
Risks and controls in this domain.
Pulled live from the Risk Library and Control Library — every entry tagged Security in the full Risk & Control spine, not a hand-maintained duplicate.
14 risks
- Adversarial evasion attacks high
- Adversarial testing coverage gap medium
- Data and model poisoning high
- Excessive agency in autonomous agents high
- Hidden context exposure medium
- Improper output handling high
- LLM application supply chain vulnerabilities high
- Misinformation and confabulation high
- Model backdoors and trojans critical
- Model extraction and inference-API abuse high
- Prompt injection high
- System prompt leakage medium
- Unbounded resource consumption medium
- Vector and embedding weaknesses medium
Frequently asked questions
What is prompt injection?
Crafted input (direct, or indirect via retrieved content) that overrides system instructions. Mitigate with instruction hierarchy and by treating any external content as untrusted data, never as instructions.
What is excessive agency in an AI system?
Giving a model or agent more permission, autonomy, or tool access than the task requires. Bound what actions are possible before you bound what actions are likely.
What is insecure output handling in an LLM application?
Treating model output as trusted, structured data when it's neither. Validate and sanitise every output before it reaches a downstream system, exactly as you would user input.
What are the five stages of a working AI security control flow?
Untrusted input, validation and isolation, the model itself (scoped to the narrowest capability the task needs), an output validation gateway, and a human-in-the-loop checkpoint for irreversible or high-consequence actions.
What evidence does a security reviewer expect for an AI system?
Control descriptions for prompt, context, and output handling; documented human-in-the-loop checkpoints for privileged actions; logging and traceability sufficient for post-incident review; and adversarial test notes for the high-risk scenarios specific to the system.
This is one of the areas Andrew advises on.
AI Security Architecture & Threat Modelling is one of five areas of expertise on the services page, grounded in exactly this control flow.