Model extraction, inversion & membership inference

Lesson 5 of 8 · 35 min · part of the AI Security Professional certification

Once a model is exposed through an API, attackers can learn about the model and about its training data purely by querying it.

Interactive exercise: What does each attack steal?

• Minimize output detail — return labels or rounded scores instead of full probability vectors.
• Rate limit & authenticate — per-key quotas, abuse detection and identity verification raise attacker cost.
• Monitor query patterns — synthetic, grid-like or high-entropy query streams are a signal.
• Differential privacy & regularization — reduce memorization, which defends against inversion and membership inference.
• Scrub training data — remove secrets and PII before training; a model can't leak what it never saw.
• Watermarking — helps prove theft after the fact.