Risk · R-164 · Prohibited & High-Risk Practices
Non-consensual intimate imagery and child sexual abuse material generation
A system is offered or used to generate intimate depictions of identifiable real people without their consent, or child sexual abuse material.
- Severity
- Critical
- Audience
- Board · Heads-of · Practitioner
- Frameworks
- Article 5(ba) · Article 5(bb) · Obscene Content
- Responsible AI domain
- Reviewed
- 2026-11-11
What it is
Two prohibitions were added to Article 5 of the EU AI Act by amendment and apply from 2 December 2026. The first covers AI systems that generate or manipulate realistic images, video, audio or similar material of an identifiable person's intimate parts, or of an identifiable person taking part in sexually explicit activity, without that person's explicit consent. The second covers systems that generate or manipulate child sexual abuse material. For providers, placing a system on the market is caught where such generation is its intended purpose, or is a reasonably foreseeable and reproducible outcome of its design that reasonable safeguards do not reliably prevent. For deployers, use is caught where the system is used for that purpose.
How it happens
- An image or video generator is released with weak or easily bypassed safeguards.
- A general model is fine-tuned or adapted so that its safeguards no longer work.
- A face-swap or editing feature accepts photographs of real people with no restriction.
- Reports of misuse are received and nothing is done to correct the system.
Why it matters
The harm to victims is severe and lasting. Creating or sharing such material is, in many of its forms, a criminal offence in the UK and elsewhere, independently of the AI Act. For a provider, the test turns on whether safeguards are reasonable and adequate and whether reported misuse is corrected, so inaction after a warning is itself exposure.
Warning signs
- A generative image, video or voice feature has had no adversarial testing for this content.
- Uploads of real people's photographs are accepted without checks.
- Misuse reports have no owner or response time.
- An openly available model with safeguards removed is in use.
Who owns it
The system owner is accountable for the safeguards. Trust and safety, or the equivalent, tests and monitors them. Legal and compliance own screening and any report to the authorities.
In practice
A photo-editing app adds a generative feature that alters clothing in uploaded pictures. Within days it is being used on photographs of real people taken from social media. The developer had tested it only on stock images. The same pattern generalises to any generative media feature released without asking how it will be misused.
Mitigating controls
Primary controls
- Content safety filtering Preventive
- Blocks generation of such material by a general-purpose system. Inputs and outputs are classified against defined harm categories by a mechanism independent of the main model, and are blocked, modified or escalated according to policy.
- Prohibited-practice screening Preventive
- Screens the proposed use against the prohibition before approval. The response to a prohibition is not to build the system. Every proposed AI use is checked against the practices the law prohibits before it is approved, the check is recorded, and it is repeated when the use changes.
Supporting controls
- Abuse monitoring Detective
- Detects and restricts users attempting it. Usage of an AI system is analysed for patterns of misuse, extraction or attack, and accounts are restricted when such patterns are detected.
- AI incident classification and playbooks Corrective
- Defines removal, reporting and escalation when such material is found. The kinds of AI failure are defined in the incident process with severity criteria, each has a playbook, and responders are trained and exercised in using them.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| EU AI Act | Article 5(ba) | Non-consensual intimate imagery The prohibition on non-consensual intimate depictions. |
| EU AI Act | Article 5(bb) | Child sexual abuse material The prohibition on child sexual abuse material. |
| NIST Generative AI Profile (NIST AI 600-1) | Obscene Content | Obscene, Degrading, and/or Abusive Content The obscene content risk. |
Sources
- EU AI Act: Regulation (EU) 2024/1689 European Union (EUR-Lex) · official text
- NIST Generative AI Profile NIST · framework
Related terms
How to cite
Axiom Verity (2026). Non-consensual intimate imagery and child sexual abuse material generation. Reference Library, R-164. https://axiom-verity.com/risks/non-consensual-intimate-imagery-generation. Reviewed 2026-11-11.
Found an error or a better source? Report a correction.