For Heads-of · Practitioner
Automation Bias
The tendency of people to over-rely on an automated system's output, accepting it without enough scrutiny even when other information suggests it is wrong. It is the main reason human oversight fails in practice, and the EU AI Act names it explicitly.
- automation bias
- human oversight
- Article 14
- meaningful human involvement
How it shows up
A reviewer who approves almost every recommendation; an override rate that falls towards zero over time; reviewers who cannot say why they agreed. Time pressure, high volumes and a system that is usually right all make it worse.
What the law says
Article 14(4)(b) of the EU AI Act requires high-risk systems to be designed so that the people overseeing them remain aware of the possible tendency to over-rely on the output. Under UK data protection law, a human review that simply rubber-stamps an automated decision is not meaningful human involvement, so the decision may count as solely automated; see automated decision-making.
Controls
Design review so the reviewer forms a view before seeing the recommendation where possible; show confidence and reasons alongside it; monitor override rates and sample agreements; and limit the volume each reviewer handles. The register entry is inadequate human oversight design; the control is human-in-the-loop review.
Sources
Related Terms
Accountability
The principle that individuals and organisations can be held responsible for decisions made by the AI systems they deploy, and must be able to explain those decisions to the people affected.
Automated Decision-Making
A decision about a person taken by technological means without meaningful human involvement. In UK data protection law, Articles 22A to 22D of the UK GDPR (inserted by the Data (Use and Access) Act 2025 and in force since 5 February 2026) allow significant automated decisions, but only with safeguards: information, the right to make representations, human intervention and the right to contest.
Explainability
The ability to give a person an account of why a system produced a particular output, in terms they can act on. Explainability is a property of the account and its audience, not of the model: an opaque model can be explainable, and an interpretable one can fail to be explained.
Human-in-the-Loop (HITL)
A design in which a person reviews, approves or can override what an AI system does before it takes effect. It is only a control if that person has the competence, information, time and authority to disagree, which is why the EU AI Act requires oversight to be designed in and UK data protection law asks whether human involvement is meaningful.