Data Poisoning

Where it happens

• Pre-training data, much of it scraped from the web, where an attacker only has to publish content likely to be collected.
• Fine-tuning data, smaller and more targeted, where a few poisoned examples can have a large effect.
• Retrieval corpora in retrieval-augmented generation, where a planted document steers answers without any retraining.
• Agent memory, where content saved in one session shapes behaviour in later ones. OWASP lists memory and context poisoning among the agentic risks.

The nearer the poisoned data sits to the deployment, the less effort the attack needs, and the more likely it is to be the deploying organisation's own problem.

Backdoors

The most dangerous form plants a backdoor: behaviour that stays dormant until a specific trigger appears in the input. Research published in January 2024 showed that a model trained to behave normally until a trigger, and harmfully after it, could keep that behaviour through standard safety training. A backdoored model can therefore pass evaluation, so the defence has to come from knowing where a model and its data came from, not only from testing what it does; see model backdoors and trojans.

How it differs from accidental data problems

Poor data quality and training data bias degrade a model without anyone intending it. Poisoning is deliberate and usually designed not to be noticed: overall metrics stay normal while a specific behaviour changes. That is why quality checks that look at averages miss it.

What good looks like

• Provenance for every dataset and model: where it came from, who could change it, and a record of what was used; see training data provenance checks.
• Integrity checks on model files and datasets through signing and hashing; see model artefact signing and integrity verification.
• Write access controlled on retrieval stores and agent memory, so untrusted parties cannot add to them.
• Targeted testing for behaviour under unusual inputs and known trigger patterns, alongside ordinary evaluation.

OWASP's 2026 list files it as data and model poisoning (LLM05), and NIST's adversarial machine learning taxonomy treats poisoning as one of its main attack classes.

Frequently asked questions

What is data poisoning? Deliberately corrupting data an AI model learns from, to degrade it or plant hidden behaviour.

What is the difference between data poisoning and a backdoor? A backdoor is one outcome of poisoning: hidden behaviour that activates only on a specific trigger.

Can testing detect data poisoning? Not reliably. A poisoned model can behave normally on ordinary tests, so provenance and integrity controls matter as much as testing.

Is RAG vulnerable to poisoning? Yes. A document planted in the retrieval corpus can steer answers without the model being retrained.

Where it sits in the register

Data and model poisoning is the register entry, with model backdoors and trojans for the hidden-trigger case and LLM application supply chain vulnerabilities for poisoned third-party models and datasets.