What it covers

AI safety spans technical work on models (alignment, robustness, interpretability, evaluation of dangerous capabilities) and the governance around them (how much autonomy a system gets, who oversees it, when it is withdrawn). AI alignment is the part concerned with whether a system pursues the goal its designers intended; safety is the broader question of whether it causes harm, aligned or not.

The frontier research questions mostly belong to model providers. What belongs to a deployer is the safety of the system it builds: the context the model is used in, the actions it can take, and the oversight around it.

Safety and security

In February 2025 the UK renamed its AI Safety Institute the AI Security Institute, and in June 2025 the US AI Safety Institute became the Center for AI Standards and Innovation. Both moves shifted official attention towards national-security misuse, such as cyber attack and chemical or biological weapons uplift.

Inside an organisation the distinction still matters. Safety asks whether the system causes harm when working as designed: a biased decision, a dangerous instruction, a flattering answer that leads a customer astray. Security asks whether someone can make it cause harm: prompt injection, data theft, misuse of an agent's tools. They need different tests and often different owners.

What a deployer can rely on

  • Developer safety frameworks. The major model developers publish policies committing to test for dangerous capabilities and to apply stronger safeguards when thresholds are crossed. They are voluntary and set by the developers themselves.
  • Pre-release testing of frontier models by the UK AI Security Institute, US CAISI and independent evaluators.
  • The International AI Safety Report, an independent scientific synthesis backed by over 30 countries, whose second edition was published in February 2026.
  • System cards, which providers publish with each model; see model card.

None of these covers a deployer's own use. Read the provider's evidence and record what it does not test.

Behaviours worth knowing

Some failure patterns recur in safety research and matter to deployers. Sycophancy is telling users what they want to hear. Specification gaming is meeting the literal objective in an unintended way. Evaluation awareness is a model behaving differently when it recognises it is being tested, which weakens the evidence safety claims rest on. And emergent capabilities are abilities that appear in a new model version without having been tested for.

What good looks like for a deployer

  • Match autonomy to evidence: give a system only as much ability to act as its testing in your context supports.
  • Test the system, not only the model, including under adversarial and unusual inputs; see AI red teaming.
  • Keep a way to stop it: oversight that can intervene, and a tested route to withdraw the system.

The register entries are inadequate human oversight design and untested AI failure scenarios; human-in-the-loop review and adversarial robustness testing are the controls.

Frequently asked questions

What is AI safety? The research and practice of making AI systems behave as intended, avoid unintended harm and stay under meaningful human control.

What is the difference between AI safety and AI security? Safety asks whether a system causes harm when working as designed. Security asks whether someone can make it cause harm.

Why did the UK AI Safety Institute change its name? In February 2025 the government renamed it the AI Security Institute to focus on national-security risks such as cyber attack and chemical and biological misuse.