Pre-deployment testing and acceptance

What it does

It makes release conditional on evidence. Acceptance criteria are agreed before testing starts, tests are run in conditions that resemble production, and the approving authority sees the results. The system owner is accountable for the decision to release; the control operates at every release, including releases of changes.

How to implement it

• Agree acceptance criteria with the business owner before testing begins, scaled to the risk tier.
• Cover accuracy on realistic cases, edge cases, safety of outputs, security, and performance across groups.
• Use test data that reflects real inputs, including poor-quality and unusual ones.
• Include testing by people outside the build team and by representative users.
• Record results, failures and any waiver with a named owner.
• Block release where a criterion is not met and no waiver is approved.

Evidence it operates

Design: a test strategy with acceptance criteria for each tier. Operation: a test report for each release showing results against criteria; waivers with approver and expiry; a release that was held when a criterion failed.

Where it fits

The gate between validation and deployment. It is the primary control for inadequate pre-deployment testing and the AI system lifecycle controls gap, and it takes inputs from adversarial robustness testing and bias testing.

Limits

Tests show the presence of faults, not their absence, and cannot anticipate every way real users will behave. Criteria written to be passed prove little. Where launch dates are fixed first, waivers become the norm and the gate stops functioning. The rate of waivers is itself worth reporting to governance.