For Heads-of · Practitioner
Data Poisoning
An attack in which an adversary deliberately injects false, misleading, or malicious data into a training dataset to compromise model performance or introduce hidden vulnerabilities.
- security
- data
- attack
- training
Definition
Unlike corrupted data (accidental), poisoning is intentional. It can cause misclassification (evasion) or insert backdoors (hidden triggers that activate under specific conditions).
Example
An attacker adds training examples to a content-moderation model that label toxic content as benign, causing the deployed model to miss harmful speech.
Mitigation
Data validation and provenance tracking, anomaly detection in training data, monitoring training metrics for unexpected shifts, and secure data pipelines.