For Heads-of · Practitioner
Data and model poisoning
Training, fine-tuning, or embedding data is deliberately manipulated to introduce vulnerabilities, backdoors, or bias into a model.
- high
- llm
- poisoning
- owasp-llm
How it happens
An adversary, or an unvetted upstream data source, introduces manipulated examples into training, fine-tuning, or embedding data, shifting the model's behaviour in a way that isn't visible from its outputs on ordinary input.
Why it matters
Poisoning attacks are designed to survive normal QA: the model behaves correctly until a specific trigger or distribution is hit, by which point it is already in production.
Mitigating controls
The controls that address this risk, ranked by effectiveness.
Training data provenance checks
Documented verification of the source, licensing, and integrity of any dataset before it is used for training or fine-tuning.
Model artifact signing and integrity verification
Cryptographic signing of trusted model artifacts and verification of that signature before a model is loaded into production.
Framework and clause references
| Framework | Clause | Title |
|---|---|---|
| MITRE ATLAS | AML.T0020 | Poison Training Data |
| OWASP Top 10 for LLM Applications | LLM05 | Data and Model Poisoning |
Related resources
The external sources behind this risk, from the Resources library.