For Heads-of · Practitioner
Adversarial robustness testing
Structured red-teaming and adversarial testing of an AI system against known attack techniques before deployment and on a recurring cadence after.
- preventive
- adversarial-ml
- red-teaming
- testing
What it does
Runs a defined battery of adversarial tests, evasion attempts, prompt injection probes, extraction queries, backdoor triggers, against a model before it ships and periodically once it is in production.
Where it fits
The direct mitigation for the adversarial-testing-coverage-gap risk, and the empirical evidence behind any claim of model robustness.
Risks this mitigates
The risks this control addresses, ranked by effectiveness.
Adversarial testing coverage gap
No structured adversarial testing exists for an AI system before or after deployment, leaving no empirical basis for its security assurance.
Model backdoors and trojans
A hidden, trigger-activated backdoor is embedded in a model via poisoned training data, weight tampering, or a payload injected into the model artifact.
Adversarial evasion attacks
Crafted adversarial input causes a model to misclassify or mis-generate, evading a downstream security or safety control.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| NIST AI Risk Management Framework (AI RMF 1.0) | Measure | Measure |
| MITRE ATLAS | AML.T0015 | Evade AI Model |