Risk · R-113 · Fairness, Bias & Impact
Missing data to test for bias
The organisation does not hold, or believes it may not use, the data on protected characteristics needed to detect discrimination, so bias goes unmeasured.
- Severity
- Medium
- Audience
- Heads-of · Practitioner
- Frameworks
- Article 4a · Lawfulness and transparency · Article 5
- Responsible AI domain
- Lifecycle stage
- Reviewed
- 2026-11-11
What it is
Testing for discrimination requires knowing who is in which group. Many organisations deliberately do not collect ethnicity, religion, disability or sexual orientation, often in the belief that this protects against discrimination or is required by data protection law. The result is that a model may discriminate through proxies while nobody is able to check. Data protection law does not forbid collecting such data for monitoring equality; it requires a lawful basis, a condition for special category data and appropriate safeguards. The EU AI Act also permits such processing for bias detection under strict conditions.
How it happens
- Characteristic data was never collected.
- It is held in a system that cannot be linked to model outcomes.
- Legal advice was to avoid holding it at all.
- Customers decline to provide it, leaving samples too small.
Why it matters
Unfairness that cannot be measured cannot be managed, and not looking is no defence to indirect discrimination. The alternative methods, such as inferring characteristics from names or postcodes, carry their own accuracy and privacy problems.
Warning signs
- Fairness testing was skipped for want of data.
- No plan exists for how group membership will be established.
- The position on collecting characteristic data has never been reviewed.
- Inferred characteristics are used with no check on their accuracy.
Mitigating controls
Primary controls
- Fairness requirements and metric selection Directive
- Identifies the data needed for testing and how it will lawfully be obtained. For each system that affects people, the relevant groups, the legal test, the fairness measures and their thresholds are defined and justified before testing begins.
Supporting controls
- Data minimisation and lawful basis review Directive
- Establishes the basis for processing characteristic data for bias testing. Before personal data is used to train, fine-tune, ground or run an AI system, a documented review confirms the lawful basis, the compatibility of the purpose and that only necessary data is used.
- Data quality and representativeness checks Detective
- Reports where group data is too sparse to support a conclusion. Training, test and reference data are assessed for accuracy, completeness, currency and coverage of the population the system will serve, before use and at each refresh.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| EU AI Act | Article 4a | Processing of special categories of personal data for bias detection and correction The provision permitting such processing for bias detection. |
| Guidance on AI and data protection | Lawfulness and transparency | Lawfulness, fairness and transparency Lawful basis for bias testing. |
| UK General Data Protection Regulation | Article 5 | Principles relating to processing of personal data Lawfulness of processing characteristic data. |
Sources
- EU AI Act: Regulation (EU) 2024/1689 European Union (EUR-Lex) · official text
- Guidance on AI and data protection UK ICO · guidance
Related terms
How to cite
Axiom Verity (2026). Missing data to test for bias. Reference Library, R-113. https://axiom-verity.com/risks/missing-data-to-test-for-bias. Reviewed 2026-11-11.
Found an error or a better source? Report a correction.