Out-of-scope responses

What it is

A general-purpose model will attempt anything it is asked. An assistant built to answer questions about a bank's current accounts can also be asked about tax, medication, a legal dispute or a competitor's products, and unless it has been constrained it will reply in the organisation's name. None of that was tested. Some of it falls into regulated territory, as unsuitable AI-generated advice or guidance describes. The risk is the gap between the purpose a system was approved for and the range of things it will actually do.

How it happens

• The system prompt describes the role and does not restrict the subject.
• Refusal behaviour was not tested at the edges of the domain.
• A user moves gradually from an in-scope question to an out-of-scope one.
• The organisation prefers an answer to a refusal, for customer satisfaction.

Why it matters

The organisation is taken to have said whatever its assistant says. Answers on untested subjects are more likely to be wrong, and the organisation may have no permission to give them. Screenshots of an assistant far outside its brief travel quickly.

Warning signs

• Transcripts show the assistant answering on unrelated subjects.
• The intended-use statement does not list what is out of scope.
• No test set probes off-topic requests.
• Refusals are counted as failures in the product's measures.