For Heads-of · Practitioner
Fairness & Bias
Also known as: algorithmic fairness · AI fairness
The principle and practice of ensuring AI systems do not produce systematically different outcomes for individuals or groups because of protected characteristics or other vulnerable attributes. Algorithmic fairness is the same idea expressed as measurable criteria.
- fairness
- bias
- discrimination
- equity
Definitions, and why they conflict
Formal definitions include demographic parity (equal positive-outcome rates across groups), equalised odds (equal error rates), calibration (a score means the same thing in every group) and individual fairness (similar cases treated similarly). Outside special cases they cannot all hold at once, so choosing which applies is a value and legal judgement for the organisation, not a technical default for the data science team.
Bias
Bias is systematic deviation from fair treatment. It can come from the training data, feature selection, the optimisation objective or the deployment context, and through proxies a model can discriminate on a characteristic it never sees. It is usually unintended and often predictable, which is why testing for it before deployment is expected rather than optional; skipping that step is inadequate fairness testing before deployment.
Regulatory landscape
The EU AI Act requires providers of high-risk systems to examine training data for possible biases and to detect, prevent and mitigate them (Article 10). The NIST AI RMF lists fairness, with harmful bias managed, among its trustworthiness characteristics, and the ICO's guidance treats fairness as a legal requirement under UK GDPR. In Great Britain, the Equality Act 2010 applies to AI-assisted decisions as to any other; see protected characteristics. Bias testing and fairness monitoring is the control.
Related Terms
Explainability
The ability to give a person an account of why a system produced a particular output, in terms they can act on. Explainability is a property of the account and its audience, not of the model: an opaque model can be explainable, and an interpretable one can fail to be explained.
Fairness Metrics
Quantitative tests of whether a model's outcomes or errors differ across groups. The main families are demographic parity, equalised odds and calibration, and outside special cases they cannot all be satisfied at once, so choosing which to use is a governance decision rather than a technical default.
Impact Assessment
A systematic evaluation of the potential positive and negative effects of deploying an AI system on individuals, communities and society. Distinct from a data protection impact assessment, which examines the lawfulness of the data processing rather than the consequences of the decision.
Protected Characteristics
The nine characteristics the Equality Act 2010 protects from discrimination in Great Britain: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, and sexual orientation. They are the legal anchor for AI fairness in the UK.
Training Data
The dataset used to train a machine learning model, from which the model learns patterns and relationships.
Related risks and controls
Disparate impact monitoring gap
The absence of ongoing monitoring for exactly this.
Inadequate fairness testing before deployment
The absence of pre-deployment testing for it.
Training data bias and discriminatory outcomes
The most common mechanism by which unfairness enters a model.
Proxy discrimination via correlated features
How unfairness persists even when a protected characteristic is excluded from the input.
Bias testing and fairness monitoring
The control covering both the testing and the monitoring gap.