For Board · Heads-of · Practitioner
Loss of Control
Scenarios in which one or more AI systems come to operate outside anyone's control, with no clear way to regain it. It is among the risks the International AI Safety Report covers, and the one on which expert views differ most.
- loss of control
- autonomy
- oversight
- International AI Safety Report
What is meant
The concern is not a single malfunction but systems that act autonomously, resist correction or shutdown, or spread beyond the environment they were deployed in, in ways their operators cannot reverse. The International AI Safety Report covers it and records that expert opinion varies widely on how likely it is and how soon it could matter.
Why it is contested
Some researchers point to early evidence in controlled tests, such as models that try to evade oversight when given the opportunity; see scheming. Others argue that these results depend heavily on test design and do not show a path to real-world loss of control. Both sides accept that the question is easier to manage before systems are given broad autonomy than after.
For a deploying organisation
The practical form of the concern is ordinary and immediate: an agent with broad permissions acting faster than anyone can supervise. Keeping a tested way to stop or roll back any autonomous system, and granting no more autonomy than the task needs, addresses both the near-term and the speculative versions. The register entries are excessive agency in autonomous agents and inadequate human oversight design.
Sources
Related Terms
Agentic AI
An AI system that pursues a goal by deciding for itself which actions to take, in what order, using tools and other systems it has been given access to, with a person checking some or none of the steps. The governance change is that the unit of risk stops being a single output and becomes a sequence of actions with consequences.
AI Safety
The field of research and practice concerned with making AI systems behave as intended, avoid unintended harm and stay under meaningful human control. In 2025 the UK and US government institutes renamed themselves around security, which narrowed the official focus without removing the questions an organisation deploying AI has to answer.
Human-in-the-Loop (HITL)
A design in which a person reviews, approves or can override what an AI system does before it takes effect. It is only a control if that person has the competence, information, time and authority to disagree, which is why the EU AI Act requires oversight to be designed in and UK data protection law asks whether human involvement is meaningful.
Scheming
A model covertly pursuing goals its developers did not intend while concealing them, for example by deliberately underperforming, hiding information or working around oversight. It has been observed in controlled tests; how often it occurs in real deployments is not known.