Fairness Metrics in AI: Parity, Equalised Odds and Calibration

The main metrics

Demographic parity: each group receives the positive outcome (an approval, an interview) at the same rate.
Equal opportunity and equalised odds: among people who should receive the positive outcome, each group does so at the same rate; equalised odds also requires equal false positive rates.
Calibration: a given score means the same probability of the outcome in every group.
Individual fairness: similar people are treated similarly, whatever their group.

Each captures a different intuition about fairness, and each can be computed from a confusion matrix broken down by group.

Why they conflict

In 2016 Kleinberg, Mullainathan and Raghavan, and separately Chouldechova, showed that when groups have different base rates of the outcome, a model cannot be calibrated and have equal error rates across groups at the same time, except in degenerate cases such as a perfect predictor. Every real deployment therefore chooses which notion of fairness to prioritise, and the choice favours some people over others.

That choice is a value and legal judgement. Left to a data scientist as a technical default, it is made without the people accountable for it.

The UK legal frame

The Equality Act 2010 does not prescribe a metric. It prohibits direct discrimination, and indirect discrimination, where a neutral rule puts a group sharing a protected characteristic at a particular disadvantage and cannot be justified as a proportionate means of achieving a legitimate aim. Metrics are the evidence for that analysis: a disparity flags a possible disadvantage, which the organisation must then justify or fix.

The EU AI Act requires providers of high-risk systems to examine their data for bias. The 2026 Digital Omnibus moved the permission to process special category data for detecting and correcting bias into a new Article 4a, widened it beyond high-risk systems, and kept it subject to strict necessity and safeguards.

What good looks like

Choose metrics deliberately, record why, and have someone accountable sign off, not only the model team.
Measure across groups even when the model never sees the characteristic, which means holding group data for testing under a proper lawful basis.
Set thresholds in advance, as part of the risk appetite, and monitor them in production.
Investigate disparities for proxies before accepting a justification.

The register entries are disparate impact monitoring gap and proxy discrimination via correlated features; the control is bias testing and fairness monitoring.

Frequently asked questions

What are fairness metrics in AI? Quantitative tests of whether a model's outcomes or error rates differ between groups, such as demographic parity, equalised odds and calibration.

Which fairness metric should we use? It depends on the decision and the harm. Because the metrics conflict and each favours different people, the choice is a governance decision.

Can a model satisfy every fairness metric? Not when groups have different base rates, except in degenerate cases. This is a mathematical result, not a limit of current techniques.

Does UK law require a particular fairness metric? No. The Equality Act 2010 sets the legal test; metrics provide the evidence for it.

Sources

Inherent Trade-Offs in the Fair Determination of Risk Scores (Kleinberg, Mullainathan and Raghavan)(opens in a new tab) arXiv · 2016-09
Fair prediction with disparate impact (Chouldechova)(opens in a new tab) arXiv · 2016-10
Equality Act 2010, section 19: Indirect discrimination(opens in a new tab) legislation.gov.uk

Related Terms

Risk & Assurance

Confusion Matrix

A table comparing a classifier's predictions with the actual outcomes, from which precision, recall and false positive and false negative rates are calculated.

Ethics & Fairness

Fairness & Bias

The principle and practice of ensuring AI systems do not produce systematically different outcomes for individuals or groups because of protected characteristics or other vulnerable attributes. Algorithmic fairness is the same idea expressed as measurable criteria.

Models & Systems

Feature Attribution

Methods that estimate how much each input contributed to a model's prediction, such as SHAP and LIME. They are standard in model validation and explanation, but they approximate the model rather than describe it, and they can be manipulated.

Risk & Assurance

Performance Metric

A quantitative measure of how well an AI model or system performs a task, such as accuracy, precision, recall, F1 score or a domain-specific measure.

Ethics & Fairness

Protected Characteristics

The nine characteristics the Equality Act 2010 protects from discrimination in Great Britain: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex, and sexual orientation. They are the legal anchor for AI fairness in the UK.

Related risks and controls

Risk

Fairness Metrics