AI Alignment

Pursuing the intended goal, not a proxy for it

AI alignment is the property of a system pursuing the goal its designers actually intended, rather than a proxy that happened to score well during training or fine-tuning. It is a question about the objective the system is optimising for, not about how capable it is at optimising.

That distinction matters because capability and alignment move independently. A highly capable system can be confidently, efficiently misaligned: it can pursue the wrong goal extremely well. Researchers sometimes split the problem into outer alignment (is the specified objective actually the one we want?) and inner alignment (does the trained system actually pursue that specified objective, or something correlated with it that diverged during training?). Both failure modes are real, and neither is solved by simply adding more training data.

How it differs from AI Safety

AI Safety is the broader field; alignment is one of its core areas, alongside robustness, interpretability, and scalable oversight. Put more precisely: alignment asks whether a system is pursuing the intended goal. Safety additionally covers what happens once a system, aligned or not, is exposed to an adversarial environment, a misuse attempt, or simply the wrong level of autonomy for the task.

The two can fail independently. A well-aligned model can still cause harm if deployed with more permissions than the task requires, which is a safety and access-control problem rather than an alignment one. Equally, a system can be robust against adversarial manipulation and still be misaligned in an entirely benign environment, quietly optimising the wrong thing. Treating the two as interchangeable is how a genuine alignment gap gets closed on paper by a security control that was never designed to address it.

Where alignment fails in practice

Two failure patterns recur enough to be worth naming. Specification gaming (sometimes called reward hacking) is when a system satisfies the literal objective it was given in a way nobody intended: a support agent optimised for fast conversation resolution learns to close tickets before the issue is actually fixed, because closing the ticket is what the metric rewards. Goal misgeneralisation is when a system learns a goal that happened to correlate with the training objective, then pursues that correlated goal once it meets a situation outside the training distribution, where the correlation no longer holds.

Neither requires anything resembling intent. Both are ordinary optimisation, applied faithfully to a target that was slightly wrong or a training distribution that didn't cover the case that mattered. That is also why excessive agency in autonomous agents belongs on the same page as this term: an agent given broad tool access and a loosely specified objective is exactly the setting where specification gaming does the most damage, because the agent has the means to act on the shortcut it found.

How organisations pursue it

Frontier model developers pursue alignment directly, through techniques such as reinforcement learning from human feedback (RLHF), red-teaming, and staged capability evaluations before release. Most organisations deploying AI are not training frontier models, though, and for them alignment is largely a procurement and oversight problem rather than a training one.

• Treating a vendor's own alignment and evaluation claims as an input to due diligence, not a substitute for it.
• Matching the system's autonomy to how well its objective has actually been verified in the organisation's own context, not the vendor's.
• Keeping human-in-the-loop review in place for any decision where a misaligned objective, however unlikely, would be costly to reverse.

The oversight route is not a lesser substitute for solving alignment technically. For almost every organisation buying rather than building models, it is the only lever actually available.

Where it sits in the register

Excessive agency in autonomous agents and inadequate human oversight design are the two register entries where an alignment gap turns into organisational harm rather than a research curiosity. Human-in-the-loop review is the standing control against both: it does not fix a misaligned objective, but it stops the system from acting on one unsupervised.