Output validation and guardrails

What it does

It treats what a model produces as untrusted input to whatever consumes it. A fixed, testable layer of ordinary code sits between the model and each consumer and rejects or neutralises anything outside what that consumer expects. The owner of each consuming system is accountable for the check; it runs on every output.

How to implement it

• Require structured output where the consumer is a program, and validate it against a strict schema.
• Encode or strip markup before rendering text in a browser or an email.
• Pass generated values to databases and shells only as parameters, never as assembled statements.
• Check tool names and arguments against an allow-list and expected ranges.
• Screen outputs for personal and restricted data before release.
• Fail closed: an output that does not validate is not delivered.

Evidence it operates

Design: for each consumer, a documented schema and the validation code path, reviewed by security. Operation: logs of rejected outputs with reasons; test results showing that outputs containing script, injected query fragments and unknown tool calls were blocked; no consumer reads model output by another route.

Where it fits

Built at design, tested before release, running on every response. It is the output-side counterpart to input filtering and limits what a successful prompt injection can do downstream.

Limits

It checks form and known-bad patterns. It cannot tell whether a well-formed output is true, appropriate or what the user wanted. A schema that is too loose passes harmful content; one that is too tight breaks legitimate use and gets relaxed.