SECAI Core · phase 11 of 15
Input manipulation
When an attacker crafts a malicious input to exploit the model's behavior
The Explain card
- Plain English
- Input manipulation is crafting a malicious input to exploit how a model behaves, whether that is a subtly altered image, a tweaked audio clip, or a carefully worded prompt.
- Example
- An attacker adds faint noise to a stop-sign photo. A human still sees a stop sign; the vehicle's vision model confidently reads "speed limit 45."
- Why it matters
- Models are decision-makers you cannot fully inspect, and they fail in ways humans would not. Defenders use adversarial testing, input validation, and redundancy so one crafted input cannot trigger a bad action.
- Hook
- Small change to the input, big change to the verdict.
Where it sits in the deck
Phase 11: Threat Landscape: Attack Vectors and Adversarial Techniques
With defences named, learn what they must defend against — the full catalogue of attack techniques targeting AI systems, their inputs, outputs, training pipelines, and supply chains.
