Training data bias and discriminatory outcomes
Historical or sampling bias in training data is reproduced or amplified in model outputs and decisions.
- high
- bias
- fairness
- high-risk
How it happens
Training or fine-tuning data encodes historical or sampling bias, which the model reproduces or amplifies in its outputs and decisions.
Why it matters
Biased outputs in a high-risk use case (credit, employment, benefits) can constitute unlawful discrimination, not just a quality defect.
Mitigating controls
The controls that address this risk, ranked by effectiveness.
Third-party model and vendor due diligence
Pre-procurement and ongoing due-diligence policy for the selection and contractual oversight of third-party AI models and datasets.
AI incident response and rollback
Defined containment, remediation, and rollback procedure for a detected AI system failure or harmful output.
AI system monitoring and logging
Continuous telemetry, drift detection, and audit logging of AI system inputs, outputs, and overrides.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| ISO/IEC 42001:2023 | Annex A.5 | Assessing impacts of AI systems |
| EU AI Act | Article 5 | Prohibited AI practices |