Inadequate independent model validation

What it is

Independent validation is review of a model by people who did not build it and have no stake in its approval. It tests whether the model is conceptually sound, whether its data is appropriate, whether it performs as claimed and where it breaks. The risk has three forms: no validation, validation without independence, and validation too shallow for what the model does. For AI systems it is often missing because the model is bought and assumed to have been validated by its supplier.

How it happens

• Developers test their own model and the result is recorded as validation.
• The validation function lacks the skills to assess machine learning or generative models.
• Vendor models are exempted on the basis of the vendor's assurance.
• Validation happens once and is not repeated after change.

Why it matters

Developers are poorly placed to find the flaws in their own assumptions. Without challenge, the organisation's knowledge of the model is limited to what its builders believed. Supervisors of banks expect validation proportionate to model risk, and its absence is an early finding in any review.

Warning signs

• The validator reports to the head of the team that built the model.
• Validation reports contain no findings.
• Bought models have no validation record.
• Models changed since their last validation remain approved.

Who owns it

The head of model risk, or the equivalent second-line function, owns validation. The board or its committee is accountable for there being such a function with adequate skill and standing.

In practice

A team builds a customer churn model and documents its own testing. The same model is later reused to decide which customers are offered retention discounts. No one outside the team has examined it, and it is found to rest on a variable that is recorded only after a customer has already left. The same pattern generalises to any model whose only reviewers are its authors.