SECAI Core · phase 14 of 15 · noun

AI threat modeling

The process of identifying potential threats and analyzing the risks in AI systems

The Explain card

Plain English
AI threat modeling is the structured process of identifying who might attack an AI system, how they could do it and what the damage would be, then prioritising the risks before building defences.
Example
Modeling a customer-facing chatbot with tool access, a team maps threats such as prompt injection via uploaded documents, training data poisoning and exfiltration through tool calls, and rates each by likelihood and impact.
Why it matters
AI systems have attack surfaces ordinary applications lack: the model, its data and its prompts. Threat modeling makes those surfaces visible so defences go where the risk is.
Hook
Draw the map before the enemy does.

Word knowledge

How the word is built, where it came from, and what it sits beside in memory.

In a sentence

A security team uses AI threat modeling before deployment to map how poisoned training data could alter a fraud-detection model.

Why these words
  • AI initialism of artificial intelligence, a term coined in English in 1955 specifies that the systems being examined use artificial intelligence
  • threat Old English thrēat, pressure, menace, or hostile force names the harmful possibilities under examination
  • modeling model plus the English action-forming suffix -ing presents the work as constructing a structured representation
Where it came from
Origin
an English compound combining AI with threat modeling, a software-security phrase established in the late 20th century
Entered the language
21st century
What changed
Threat modeling moved from general software and network security into AI security, while AI narrowed the phrase to systems involving trained models and machine learning.
How it is spelled
Pattern
AI remains uppercase because it is an initialism
Pattern
threat begins with the consonant digraph th
Pattern
modeling keeps one l in standard American spelling before -ing
Breaks the pattern
British spelling commonly doubles the l in modelling

Spelled like

  • traveling
  • labeling
  • modeler
Broken into chunks
  • AI
    • AI safety
    • AI security
  • threat
    • threat actor
    • threat intelligence
  • model
    • modeler
    • model-based
  • -ing
    • testing
    • monitoring
What it sits beside

Same subject

Same shape

  • risk modeling
  • security testing
  • attack modeling

AI attack techniques

security analysis practices

  • risk assessment
  • attack surface analysis
  • penetration testing

Where it sits in the deck

Phase 14: Incident Response, Evaluation, and Knowledge Bases

When detection fires, teams need structured response plans, scoring frameworks, and curated knowledge bases to triage, measure, and learn from AI security events.

Neighbours in this phase