Control · C-068 · Data & Data Governance
Data quality and representativeness checks
Training, test and reference data are assessed for accuracy, completeness, currency and coverage of the population the system will serve, before use and at each refresh.
- Control type
- Detective
- Family
- Risks addressed
- 7
- Audience
- Heads-of · Practitioner
- Frameworks
- Article 10 · Annex A.7 · Principle 3
- Responsible AI domain
- Lifecycle stage
- Reviewed
- 2026-11-11
What it does
It finds defects in data before a model learns them. Each dataset is measured against defined standards: whether records are correct, whether fields are populated, whether the data is current, whether labels are consistent, and whether the people and situations the system will meet are adequately present. Results are recorded and shortfalls are fixed, accepted or cause the data to be rejected. The data owner is accountable for quality; the model owner for judging fitness for the purpose.
How to implement it
- Define quality measures and thresholds for each dataset by its intended use.
- Profile the data: missing values, duplicates, outliers, label consistency, date ranges.
- Compare the make-up of the data with the population to be served, by relevant groups and segments.
- Examine how the data was produced, including past processes that shaped the labels.
- Check retraining data for the model's own earlier outputs.
- Record results and decisions, and repeat at every refresh.
Evidence it operates
Design: data quality standards and a checking procedure. Operation: a quality report for each dataset in use; comparisons with the served population; records of remediation or acceptance; a dataset rejected or supplemented as a result.
Where it fits
After provenance, before training and evaluation. It is the primary control for poor quality or unrepresentative training data, training data bias and self-reinforcing feedback loops. Its findings are recorded through dataset documentation.
Limits
Checks find the defects someone thought to measure. Representativeness needs data on group membership, which may not be held and may be unlawful to collect without care. A dataset can be accurate, complete and representative of a past that was itself unfair. Quality is relative to purpose, so a dataset passed for one use is not thereby fit for another.
Risks this mitigates
Primary mappings
- Poor quality or unrepresentative training data High
- Assesses accuracy, completeness, currency and coverage before use and on refresh. Training or reference data is inaccurate, incomplete, out of date or unrepresentative of the population the system serves, and the model inherits the defect.
- Self-reinforcing feedback loops Medium
- Checks retraining data for the model's own influence. A model's outputs shape the data it is later trained or evaluated on, entrenching early errors and narrowing outcomes over time.
- Training data bias and discriminatory outcomes High
- Checks whether the data represents the population the system will serve. Bias in the data a model learns from, whether from history, sampling or labelling, is reproduced or amplified in its outputs and in the decisions that rely on them.
- Unequal performance across groups High
- Confirms each group is adequately represented in training and test data. A system is less accurate or less useful for some groups, languages, accents or dialects than for others, although everyone is treated by the same process.
Supporting mappings
- Data and model poisoning High
- Statistical checks on new data surface poisoned or anomalous records. Training, fine-tuning or embedding data is manipulated so that the resulting model carries a vulnerability, a hidden behaviour or a bias its developers did not intend.
- Degraded information environment Low
- Screens incoming data for synthetic content. The growing volume of synthetic content lowers the reliability of the public and internal information that the organisation and its models depend on.
- Missing data to test for bias Medium
- Reports where group data is too sparse to support a conclusion. The organisation does not hold, or believes it may not use, the data on protected characteristics needed to detect discrimination, so bias goes unmeasured.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| EU AI Act | Article 10 | Data and data governance Data governance. |
| ISO/IEC 42001:2023 | Annex A.7 | Data for AI systems Data for AI systems. |
| SS1/23: Model Risk Management Principles for Banks | Principle 3 | Model development, implementation and use Data quality. |
Related terms
How to cite
Axiom Verity (2026). Data quality and representativeness checks. Reference Library, C-068. https://axiom-verity.com/controls/data-quality-and-representativeness-checks. Reviewed 2026-11-11.
Found an error or a better source? Report a correction.