Miscalibrated confidence

What it is

A well-calibrated system is right about as often as its stated confidence implies. Many are not: a classifier's high scores may be wrong more often than they suggest, and a language model writes with the same assurance whether it is correct or inventing. Calibration matters because people and processes use confidence to decide what to review.

How it happens

• Scores are treated as probabilities without having been tested as such.
• A generative model's fluent tone is read as certainty.
• Confidence thresholds were set once and never checked against outcomes.
• The interface shows an answer and hides the uncertainty.

Why it matters

Review effort is directed by confidence. If confidence is wrong, the cases that needed attention are passed and effort is spent elsewhere. It also feeds over-reliance, since a system that never appears unsure is trusted more than it should be.

Warning signs

• High-confidence outputs are wrong at a noticeable rate.
• No calibration analysis appears in the validation record.
• The system never declines to answer or expresses doubt.
• Referral thresholds have no documented basis.