AI Risk & Governance Glossary

Definitions of AI governance, compliance, security and safety terms, each linked to the risks and controls it bears on. Entries marked In depth are full reference articles.

120 terms, 38 in depth

Governance

AI Inventory

A maintained register of every AI system, model and AI-enabled feature an organisation uses, builds or buys, with an owner, a purpose, a risk tier and a status for each.

In depth

AI TRiSM (AI Trust, Risk and Security Management)

Gartner's name for the discipline of making AI systems trustworthy, governable and secure in operation: governance, runtime inspection and enforcement, information governance, and the infrastructure underneath.

In depth

Audit Trail

A complete, timestamped record of the actions an AI system took and the decisions made with it, so that they can be reviewed, explained and attributed afterwards.

Control Objective

A specific, measurable statement of what a control is meant to achieve against a named risk.

In depth

Governance Framework

A structured set of policies, roles and controls that guides an organisation's decisions about AI and its oversight of AI systems.

Human-in-the-Loop (HITL)

A design in which a person reviews, approves or can override what an AI system does before it takes effect.

In depth

Model Card

A document describing a machine learning model's intended uses, performance, limitations and known risks, so that people deciding whether and how to use it can do so on evidence.

In depth

Risk Appetite

The level and types of risk an organisation is willing to accept in pursuit of its strategic objectives.

In depth

Shadow AI

The use of AI tools, models or features inside an organisation without the knowledge, approval or oversight of whoever is responsible for governing them.

In depth

Trustworthy AI

The umbrella term for AI that people can justifiably rely on.

Compliance & Regulation

AI Cyber Security Code of Practice

The UK government's voluntary code, published in January 2025, setting 13 principles for securing AI systems across five lifecycle phases.

AI Literacy

The skills, knowledge and understanding that let a person use, oversee or be affected by an AI system with an informed view of what it can do, where it fails and what it means for them.

In depth

AI Office

The European Commission body that supervises general-purpose AI models under the EU AI Act and coordinates the Act's enforcement with national authorities.

AI Regulatory Sandbox

A controlled environment in which a firm can develop or test an AI system under a regulator's supervision, with guidance on how the rules apply.

Automated Decision-Making

A decision about a person taken by technological means without meaningful human involvement.

In depth

Conformity Assessment

The procedure by which the provider of a high-risk AI system shows that it meets the EU AI Act's requirements before placing it on the market.

Consumer Duty

The FCA's outcomes-based standard requiring firms to deliver good outcomes for retail customers, in force since July 2023.

Content Provenance

Verifiable information about where a piece of content came from and how it was made or changed, attached to the content itself.

Critical Third Parties Regime

The UK regime under which HM Treasury can designate a third party whose services are critical to the financial sector, bringing it under direct oversight by the Bank of England, the PRA and the FCA.

Digital Omnibus on AI

Regulation (EU) 2026/1744, the 2026 law that amended the EU AI Act.

EU AI Act

The European Union's regulation of AI systems by risk category, in force since 1 August 2024 and applying in stages.

In depth

Explainability

The ability to give a person an account of why a system produced a particular output, in terms they can act on.

In depth

General-Purpose AI Model

An AI model that displays significant generality and can competently perform a wide range of distinct tasks, however it is placed on the market.

In depth

High-Risk AI System

An AI system that the EU AI Act subjects to its full set of requirements, either because it is a safety component of a product covered by EU product law (Annex I) or because it is used in one of eight listed areas (Annex III), which include credit scoring and life and health insurance pricing.

In depth

Impact Assessment

A systematic evaluation of the potential positive and negative effects of deploying an AI system on individuals, communities and society.

In depth

ISO/IEC 23894

International guidance, published in 2023, on managing risk related to AI.

ISO/IEC 42001

The international standard for an AI management system (AIMS): the policies, roles, risk assessment, impact assessment, controls and evidence an organisation puts around its development and use of AI so that the whole can be audited and certified.

In depth

ISO/IEC 42005

The international standard, published in May 2025, giving guidance on how to carry out an AI system impact assessment: identifying the people and groups a system affects and assessing its potential benefits and harms.

NIST AI Risk Management Framework

The US National Institute of Standards and Technology's voluntary framework for managing AI risk, built around four functions: Govern, Map, Measure and Manage.

In depth

Post-Market Monitoring

A provider's documented, continuous collection and analysis of data on how its high-risk AI system performs once in use, so that problems are found and fixed throughout the system's life.

PRA SS1/23

The Prudential Regulation Authority's supervisory statement setting out five principles for model risk management at UK banks.

Prohibited AI Practices

The uses of AI that Article 5 of the EU AI Act bans outright.

Provider and Deployer

The two main roles the EU AI Act assigns obligations to.

In depth

Transparency

The disclosure of information about an AI system to the people who need it: that AI is being used, what it does, what it is for and what risks it poses, in terms they can understand.

Risk & Assurance

A/B Testing

An experimental method that compares two versions of a system (A and B) by sending each to a random subset of users, to establish which performs better.

AI Assurance

The work of measuring, evaluating and communicating whether an AI system, or the organisation running it, meets the claims made for it, so that others can rely on those claims.

In depth

AI Evaluation

Systematic measurement of what an AI model or system can do, how it tends to behave and how well its safeguards hold, using benchmarks, task-based tests, red teaming or studies with people.

AI Red Teaming

Structured adversarial testing in which testers try to make an AI system fail, either by producing harmful or prohibited output (safety red teaming) or by being subverted to act against its owner (security red teaming).

In depth

AI TPRM (AI Third-Party Risk Management)

A phrase with two meanings that are often confused: using AI tools to run a third-party risk management (TPRM) programme, and managing the third-party risk that comes from buying AI.

In depth

Confusion Matrix

A table comparing a classifier's predictions with the actual outcomes, from which precision, recall and false positive and false negative rates are calculated.

Impact Tolerance

The maximum tolerable level of disruption to an important business service, set under the UK operational resilience regime.

Model Risk Management (MRM)

The discipline of identifying, measuring, controlling and reporting the risk that a model is wrong, misused or misunderstood, across its whole life from development to retirement.

In depth

Performance Metric

A quantitative measure of how well an AI model or system performs a task, such as accuracy, precision, recall, F1 score or a domain-specific measure.

Safety Case

A structured argument, supported by evidence, that a system is acceptably safe for a given use in a given context.

Testing AI Systems

Systematic evaluation of AI models and systems to verify that they perform as intended, identify failure modes and assess risks.

Third-Party Risk

The exposure an organisation carries through dependencies on external parties who build, train, host or operate the AI systems it uses.

In depth

Three Lines of Defence

The risk governance model used across UK financial services, in which the business owns and manages risk, an independent risk and compliance function oversees it, and internal audit gives the board independent assurance.

Security

Adversarial Attack

An attempt to manipulate or deceive an AI model by deliberately crafting inputs designed to cause misclassification or unintended behaviour.

Agent Goal Hijack

Redirecting an AI agent's objective or plan, usually by placing instructions in content the agent reads, so that it pursues the attacker's goal with its own tools and permissions.

Agent2Agent Protocol (A2A)

An open protocol for AI agents from different vendors and frameworks to discover each other, describe what they can do and delegate tasks to one another.

AI Bill of Materials

A machine-readable inventory of what an AI system is built from: models, datasets, adapters, software dependencies, and their provenance and licences.

AI Supply Chain Security

Assuring the integrity and provenance of everything an AI system is built from and depends on: pre-trained models and weights, adapters, datasets, libraries, serving infrastructure, plug-ins and tools.

In depth

Data Poisoning

An attack in which an adversary deliberately corrupts data an AI model learns from, to degrade it or plant hidden behaviour.

In depth

Deepfake

AI-generated or manipulated image, audio or video content that resembles real people, objects, places or events and would falsely appear to a person to be authentic.

Excessive Agency

Giving an LLM-based system more functions, permissions or autonomy than its task needs, so that a manipulated or mistaken output becomes a damaging action.

Guardrails

Runtime checks on an AI system's inputs and outputs, such as classifiers, filters and policy rules, that block, modify or flag content before it reaches the model or after it leaves.

Hidden Context Exposure

The exposure of content an AI application placed in a model's context on the assumption users could not see it: system prompts, retrieved documents, schemas, policy logic or other users' data.

Jailbreak

A technique for inducing a language model to produce output that its safety training was intended to refuse.

In depth

Lethal Trifecta

Simon Willison's name for the combination of three capabilities that makes an AI agent exploitable for data theft: access to private data, exposure to untrusted content, and the ability to communicate externally.

Membership Inference

An attack that works out whether a particular person's record was in a model's training data by examining how the model responds.

Memory Poisoning

Corrupting what an AI agent remembers or retrieves, so that later sessions, possibly other users' sessions, behave as the attacker wants.

MITRE ATLAS

MITRE's knowledge base of the tactics and techniques adversaries use against AI systems, modelled on the MITRE ATT&CK framework for conventional cyber attacks and built from real-world case studies and red-team findings.

Model Backdoor

Hidden behaviour planted in an AI model that stays dormant until a specific trigger appears in the input, then makes the model act as the attacker intends.

Model Context Protocol (MCP)

An open protocol for connecting AI applications to external tools and data through servers that expose tools, resources and prompts.

In depth

Model Extraction

Reconstructing a model's behaviour or parameters by querying it repeatedly and learning from the answers, to copy its intellectual property or to build a surrogate for planning further attacks.

Model Inversion

An attack that uses a model's outputs to reconstruct information about its training data, such as recovering a recognisable face from a facial recognition model, or sensitive attributes of the people in a dataset.

Non-Human Identity

Any identity used by software rather than a person to authenticate to other systems, such as service accounts, API keys and tokens.

OWASP Top 10 for Agentic Applications

OWASP's list of the ten most critical security risks specific to autonomous AI agents, published in December 2025.

OWASP Top 10 for LLM Applications

The OWASP GenAI Security Project's ranked list of the ten most critical security risks in applications built on large language models.

In depth

Prompt Injection

An attack in which untrusted content reaching a language model is interpreted as instructions rather than data, causing the model to act on an attacker's intent instead of the operator's.

In depth

Slopsquatting

Registering software package names that AI coding assistants tend to invent, so that a developer who installs a suggested but non-existent package installs the attacker's code instead.

AI Safety

AI Alignment

The property of an AI system pursuing the goal its designers actually intended, rather than a proxy that happened to score well during training.

In depth

AI Safety

The field of research and practice concerned with making AI systems behave as intended, avoid unintended harm and stay under meaningful human control.

In depth

Chain-of-Thought Monitoring

Reading the intermediate reasoning a reasoning model produces before its answer, to detect intent to misbehave.

Frontier AI

The most capable general-purpose AI models, at or near the leading edge.

In depth

Frontier Safety Framework

A frontier AI developer's published commitment to test its models for dangerous capabilities and to apply stronger safeguards, or pause, when defined capability thresholds are crossed.

Loss of Control

Scenarios in which one or more AI systems come to operate outside anyone's control, with no clear way to regain it.

Reward Hacking

A system meeting the literal objective it was trained or rewarded on in a way its designers did not intend, such as gaming a test rather than solving the task.

Sandbagging

Strategic underperformance on an evaluation, so that a model appears less capable than it is.

Scheming

A model covertly pursuing goals its developers did not intend while concealing them, for example by deliberately underperforming, hiding information or working around oversight.

Sycophancy

A model's tendency to tell users what they want to hear, agreeing, flattering or validating at the expense of accuracy.

Uplift

The additional ability an AI system gives a person to do something, compared with what they could do without it, for example using search engines and textbooks.

Ethics & Fairness

Accountability

The principle that individuals and organisations can be held responsible for decisions made by the AI systems they deploy, and must be able to explain those decisions to the people affected.

Automation Bias

The tendency of people to over-rely on an automated system's output, accepting it without enough scrutiny even when other information suggests it is wrong.

Contestability

The ability of a person affected by an AI-assisted decision to challenge it and have it reconsidered by someone with the authority to change it.

Counterfactual Explanation

An explanation of an automated decision that states the smallest change to the inputs that would have changed the outcome, such as an application that would have been approved with a higher declared income.

Fairness & Bias

The principle and practice of ensuring AI systems do not produce systematically different outcomes for individuals or groups because of protected characteristics or other vulnerable attributes.

Fairness Metrics

Quantitative tests of whether a model's outcomes or errors differ across groups.

In depth

Protected Characteristics

The nine characteristics the Equality Act 2010 protects from discrimination in Great Britain: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, and sexual orientation.

Models & Systems

Agentic AI

An AI system that pursues a goal by deciding for itself which actions to take, in what order, using tools and other systems it has been given access to, with a person checking some or none of the steps.

In depth

Context Window

The maximum length of text (measured in tokens) that a language model can consider at once when generating a response.

Embedding

A numerical representation of text, images or other data in a high-dimensional space, where items with similar meaning sit close together.

Emergent Capabilities

Abilities that appear in larger AI models but were not explicitly trained for and were not evident in smaller versions.

Feature Attribution

Methods that estimate how much each input contributed to a model's prediction, such as SHAP and LIME.

Fine-Tuning

The process of adapting a pre-trained model to a specific task or domain by training it further on a smaller, task-specific dataset.

Hallucination

Output that is fluent, confident and false.

In depth

Large Language Model (LLM)

A deep learning model trained on very large amounts of text, able to generate and interpret human-like language across a wide range of topics and tasks.

Machine Learning

A branch of artificial intelligence in which systems learn patterns from data rather than being explicitly programmed for every scenario.

Mechanistic Interpretability

Research that tries to reverse-engineer what a neural network computes internally, identifying the features it represents and the circuits that connect them, in terms a person can understand.

Model Collapse

The degradation in quality and diversity that occurs when language models are trained, generation after generation, on content generated by other language models.

Model Interpretability

An intrinsic property of a model: the degree to which a person can understand the mechanism by which it turns inputs into outputs.

In depth

Multimodal AI

AI systems that process and combine more than one type of data (text, images, audio, video) to understand and respond to the world.

Neural Network

A computational model loosely inspired by biological neurons, made of interconnected layers of artificial nodes that learn to recognise patterns in data.

Prompt Engineering

The practice of crafting and refining prompts to steer language models towards the outputs wanted, treating the prompt itself as a design problem.

Reinforcement Learning from Human Feedback (RLHF)

A training method in which people rate a model's outputs, a reward model learns to predict those ratings, and the model is then fine-tuned to produce outputs the reward model scores highly.

Retrieval-Augmented Generation (RAG)

A technique in which a language model retrieves relevant documents or data before generating a response, grounding its output in source material rather than in its training data alone.

Token

A unit of text that a language model processes, typically a word, part of a word or a character.

Transformer

A deep learning architecture that uses self-attention to process sequences of data, and the foundation of modern large language models.

Data

Data Quality

The degree to which data is accurate, complete, consistent and representative of what it is meant to measure.

Differential Privacy

A mathematical framework for limiting what can be learned about any individual from an algorithm's output, by adding carefully calibrated noise so that the output barely changes whether or not that individual's data was included.

Federated Learning

A machine learning approach in which a model is trained across many separate data sources without the data itself being gathered in one place.

Synthetic Data

Artificially generated data, created by algorithms or models, used instead of or alongside real data in training and testing.

Training Data

The dataset used to train a machine learning model, from which the model learns patterns and relationships.