SECAI Core · phase 11 of 15
Model inversion attack
A security attack where an attacker trains the model with a specific set of records and uses it to inquire the data used to train the model
The Explain card
- Plain English
- A model inversion attack uses a model's responses to reconstruct or infer the data it was trained on. The attacker probes with crafted queries and works backwards from the outputs.
- Example
- By repeatedly querying a face recognition API and optimizing inputs against its confidence scores, an attacker reconstructs a recognizable image of a person in the training set.
- Why it matters
- Models can leak their training data through their answers alone. Defenders limit confidence-score exposure, rate-limit queries, and apply privacy-preserving training techniques.
- Hook
- Running the model in reverse to see what it saw.
Where it sits in the deck
Phase 11: Threat Landscape: Attack Vectors and Adversarial Techniques
With defences named, learn what they must defend against — the full catalogue of attack techniques targeting AI systems, their inputs, outputs, training pipelines, and supply chains.
