Independent model validation

What it does

It provides informed challenge by people with no stake in the model's approval. Validation asks whether the approach is sound, whether the data is fit, whether the claimed performance can be reproduced, and where the model fails, and it reports findings that must be resolved or accepted before use. The head of model risk owns it. It operates before first use, after material change and on a cycle set by tier.

How to implement it

• Establish a validation function that reports separately from model development.
• Set the scope and depth of validation for each risk tier.
• Reproduce key results independently, on data the developers did not use.
• Test stability, sensitivity, performance by segment and behaviour at the limits.
• Apply the same standard to bought models, using the supplier's evidence as an input.
• Issue findings with severity, an owner and a date, and track them.

Evidence it operates

Design: a validation policy with tiers, scope and independence requirements. Operation: a validation report for each model in use, dated and with findings; evidence that approval followed validation; revalidation records after change; an open findings log.

Where it fits

Sits between build and approval, and recurs through the model's life. It is the primary control for inadequate independent model validation and model not fit for its intended purpose, and depends on model and system documentation.

Limits

Validation is bounded by the validators' skill and access. For a hosted model with no access to weights or training data, it can test behaviour and not construction. A function without standing to delay a launch provides comment, not control.