Defending against data poisoning

Lesson 4 of 8 · 35 min · part of the AI Security Professional certification

Data poisoning is the intentional corruption of training or fine-tuning data to influence a model's behaviour. Because modern models train on huge, often scraped datasets, attackers may only need to control a tiny fraction of the data.

A backdoor is a targeted form of poisoning: the model behaves normally on almost all inputs, but a specific trigger (a sticker, a rare phrase, a pixel pattern) makes it produce the attacker's chosen output.

• Provenance — know where every dataset came from; prefer signed, versioned, trusted sources.
• Integrity — hash and verify datasets and models; protect training pipelines like production.
• Sanitization — dedupe, detect label noise and outliers, filter suspicious clusters.
• Adversarial evaluation — test for triggers and targeted behaviours, not just average accuracy.
• RAG counts too — poisoning a vector database or knowledge base is poisoning at inference time (OWASP LLM08).

Interactive exercise: Secure the training data pipeline