Poor generalisation outside training conditions

What it is

A model interpolates within what it has seen. Asked about something different, it still answers, and the answer may be arbitrary. The difference from drift is that this is about individual inputs at any time, including the first day: the rare case, the new product, the customer who fits no pattern. An overfitted model, tuned too closely to its training data, generalises worst.

How it happens

• An input falls outside the ranges present in training.
• A combination of ordinary values occurs that the model never met.
• The format or quality of an input differs, such as a poor scan or an unfamiliar dialect.
• The model was fitted too closely to incidental features of its training set.

Why it matters

Unusual cases are often the important ones: the large transaction, the vulnerable customer, the edge of policy. A model that fails silently there concentrates its errors where they cost most.

Warning signs

• No test covers inputs at or beyond the edges of the training data.
• The system has no means of declining or referring a case.
• Errors cluster in small or new segments.
• Training accuracy is far above test accuracy.

Who owns it

The model owner is accountable for knowing the model's limits. Engineering implements detection of unfamiliar inputs and referral. The business owner decides what happens to referred cases.

In practice

A property valuation model trained on suburban sales is used for a remortgage on a listed building in a remote area. It returns a figure with the same confidence as any other. The same pattern generalises to any model applied to cases outside the range it learned from.