For Heads-of · Practitioner

Adversarial testing coverage gap

No structured adversarial testing exists for an AI system before or after deployment, leaving no empirical basis for its security assurance.

medium
adversarial-ml
red-teaming
testing-gap

How it happens

An AI system goes through functional QA but is never subjected to structured red-teaming or adversarial testing against the techniques catalogued in frameworks like MITRE ATLAS, because that testing discipline is assumed to be the model vendor's job, not the deploying organisation's.

Why it matters

Without adversarial testing, every other control in this category is unverified: the organisation is asserting resilience it has never actually tried to break.

Mitigating controls

The controls that address this risk, ranked by effectiveness.

primary

Adversarial robustness testing

Structured red-teaming and adversarial testing of an AI system against known attack techniques before deployment and on a recurring cadence after.

Framework and clause references

Framework	Clause	Title
NIST AI Risk Management Framework (AI RMF 1.0)	Measure	Measure
ISO/IEC 42001:2023	Clause 9	Performance evaluation

Related resources

The external sources behind this risk, from the Resources library.

MITRE · framework

MITRE ATLAS

NIST · framework

NIST AI Risk Management Framework (AI RMF 1.0)

Back to the risk library