Every SECAI Core term, in study order

245 terms across 15 phases, in the order the deck teaches them. Each one gets a page you can read in a minute and a quiz that checks whether you did.

A researcher stands with their back to us at a glass wall scrawled with neural network diagrams and equations in marker, city lights burning through it in a dark office.

What AI Is: Core Concepts and Paradigms

Phase 1 · 13 terms

You cannot secure, govern, or attack something you cannot define — establish what AI actually is before any other concept can land.

  1. 1 Artificial intelligence The science of creating machines with the ability to develop problem-solving and analysis strategies without significant...
  2. 2 Machine learning A component of AI that enables a machine to develop strategies for solving a task given a labeled dataset where features...
  3. 3 Statistical learning Developing mathematical models to explain and predict data behavior
  4. 4 Supervised learning A category of machine learning that uses labeled data when training algorithms to predict outcomes.
  5. 5 Unsupervised learning A category of machine learning that does not use labeled data to predict outcomes, but instead relies on the computer to...
  6. 6 Reinforcement learning A type of machine learning that uses trial and error to make improved decisions by iterating through possible solutions...
  7. 7 Deep learning A refinement of machine learning that enables a machine to develop strategies for solving a task given a labeled dataset...
  8. 8 Natural language processing The use of statistical techniques, modeling, and machine learning tools by computers to process text-based datasets and...
  9. 9 Generative AI A class of artificial intelligence models designed to generate novel data (such as text, images, audio, or code) by lear...
  10. 10 Chatbot A software application that uses AI to understand user messages and generate conversational responses, allowing people t...
  11. 11 Personal assistant An AI-powered application that helps an individual manage tasks and information—such as scheduling, reminders, messages,...
  12. 12 Automation Use of scripts and APIs to provision and deprovision systems without manual intervention.
  13. 13 Central processing unit Principal microprocessor in a PC or mobile device responsible for running firmware, operating system, and applications s...
A close look at a GPU accelerator seated in an open server chassis, heatsink fins and power cabling sharp in front, training loss curves going soft on a screen behind.

How Models Are Built: Architectures and Learning Mechanics

Phase 2 · 12 terms

With the paradigms named, zoom in on the architectural building blocks that turn data into trained artifacts.

  1. 14 Transformer A neural network architecture that excels at processing and analyzing sequences of data by focusing on the connections b...
  2. 15 Convolutional neural network A deep learning method designed to process and analyze visual data by mimicking the way the human brain processes images...
  3. 16 Autoencoder Small neural networks trained to recreate the records they see during learning
  4. 17 Generative adversarial network A type of artificial intelligence model in which two neural networks compete against each other to generate realistic sy...
  5. 18 Large language model An advanced artificial intelligence model trained on vast amounts of text data to understand, generate, and analyze huma...
  6. 19 Small language model A language model with a relatively low number of parameters and computational requirements, designed to run efficiently...
  7. 20 Embeddings Numerical representations of real-world entities utilized by machine learning and artificial intelligence systems to com...
  8. 21 Pre-trained model An AI model that has already been trained on a large, general dataset and can be reused or fine-tuned for specific tasks
  9. 22 Fine tuning Training a neural network model with a smaller dataset tailored for a specific task
  10. 23 Federated learning A machine learning approach where a central model is trained collaboratively across many decentralized devices or server...
  11. 24 Differentially private stochastic gradient descent A method used for machine learning data encryption to protect data privacy of the data endpoints by calculating a gradie...
  12. 25 Pruning Removing less important elements of the model that have negligible impact on performance
A data engineer's desk from overhead: two monitors of tabular data and a pipeline graph, a notebook of column notes, hands on the keyboard and a coffee that went cold hours ago.

Data Fundamentals: Types, Pipelines, and Preparation

Phase 3 · 14 terms

Models are only as good as their data — understand the raw material and the engineering that shapes it before covering how it flows or breaks.

  1. 26 Structured data Data that is organized and stored in tables, in rows and columns.
  2. 27 Unstructured data Data that are not able to be stored in columns and rows, such as audio and video signals, large bodies of text, and webp...
  3. 28 Semi-structured data Data that is a mix of both structured and non-structured data.
  4. 29 Data storage The hardware and software systems used to save, organize, and manage the data needed for training, validating, deploying...
  5. 30 Vector storage A lookup table that turns pieces of text into numeric fingerprints
  6. 31 Data processing The set of steps used to collect, clean, transform, and organize raw data into a structured, usable form
  7. 32 Data cleaning The process of ensuring that the data used in analysis is of a high-enough quality that it gives decision makers confide...
  8. 33 Deduplication A technique for removing duplicate copies of repeated data. In SIEM, the removal of redundant information provided by se...
  9. 34 Data augmentation A process of generating additional, artificial data that is similar to existing data, primarily for the purpose of train...
  10. 35 Data balancing A technique used to realign a training set so that rare, yet critical events receive proportionate attention
  11. 36 Data profiling Determining the volume of data, the types of data and quality of the data.
  12. 37 Edge case A rare, unusual, or extreme input or situation that lies at the boundary of what the AI model was trained on or usually...
  13. 38 Data engineer A role responsible for building and maintaining the pipelines that acquire, clean, label, and deliver data with appropri...
  14. 39 Data scientist A role that explores data to define a problem, select relevant features, and develop baseline models that demonstrate va...
Gloved hands close a numbered evidence box on a steel table, a signed chain of custody form and a wax sealed folder waiting beside it under cold overhead light.

Data Integrity, Governance, and Provenance

Phase 4 · 14 terms

Once you know what data is, learn the principles that keep it trustworthy, traceable, and minimized throughout its lifecycle.

  1. 40 Data integrity To safeguard the trustworthiness and accuracy of data throughout its lifecycle
  2. 41 Data provenance A record of the origin, history, and transformations of data
  3. 42 Data lineage A record of the origin, movement, and transformation of data as it flows through systems and processes.
  4. 43 Data verification The process of confirming that the data is accurate or true.
  5. 44 Data governance The overall management of the availability, usability, and security of the information used in an organization.
  6. 45 Data minimization In data protection, the principle that only necessary and sufficient personal information can be collected and processed...
  7. 46 Data access Control of which data a user or entity within the AI system has access to
  8. 47 Data in transit Information that is being transmitted between two hosts, such as over a private network or the Internet. Also referred t...
  9. 48 Data in use Information that is present in the volatile memory of a host, such as system memory or cache.
  10. 49 Chain of custody Record of handling evidence from collection to presentation in court to disposal.
  11. 50 Personally identifiable information Data that can be used to identify or contact an individual (or, in the case of identity theft, to impersonate them).
  12. 51 Protected/personal health information Information that identifies someone as the subject of medical and insurance records, plus associated hospital and labora...
  13. 52 Intellectual property Data that is of commercial value and can be granted rights of ownership, such as copyrights, patents, and trademarks.
  14. 53 Privacy An AI principle that AI systems collect, use, share, and store personal data in a way that minimizes data collection, pr...
Over someone's shoulder as they type a prompt into a chat window late at night, API documentation glowing on the second screen.

Interacting with Models: Prompting, APIs, and Retrieval

Phase 5 · 17 terms

With a working model and clean data, learn the interaction layer — how users and systems communicate instructions and retrieve grounded answers.

  1. 54 Application programming interface Methods exposed by a script or program that allow other scripts or programs to use it. For example, an API enables softw...
  2. 55 Plugins Additions that help tailor the software product to more closely match the infrastructure being managed from one organiza...
  3. 56 Model context protocol AI systems, agents, and tools to share data about memory, goals, and intermediate results, enabling multi-agent systems,...
  4. 57 System prompt The background instructions in an AI system that define the model's behavior, tone, or restrictions
  5. 58 System role A compact statement that defines the model's persona
  6. 59 User prompt The instructions, questions, or examples a user enters into an AI system to specify what task it should perform and how...
  7. 60 Prompt engineering The disciplined practice of directing large-language models through clear and precise instructions
  8. 61 Prompt template The predefined structures used to format user queries or system instructions before they are processed by an AI model
  9. 62 Prompt compression The reduction of the complexity of a prompt while preserving its meaning and required information in order to reduce cos...
  10. 63 Zero-shot prompting An AI prompting technique where the model is asked to perform a task with no prior examples in the prompt, only an instr...
  11. 64 One-shot prompting An AI prompting technique where one example input-output pair is included in the prompt before the real query so the mod...
  12. 65 Multi-shot prompting An AI prompting technique where several example input-output pairs are provided in the prompt before the real query so t...
  13. 66 Retrieval-augmented generation A technique where a generative model retrieves relevant information from an external knowledge source and then uses that...
  14. 67 Source grounding The technique of connecting an AI model's outputs to specific, retrieved, and verifiable external information sources, a...
  15. 68 Grounding check A verification step that tests whether the AI model's outputs are correctly supported by specified sources or evidence (...
  16. 69 No-code When a tool or platform is able to produce code output with no human involvement in writing of the code
  17. 70 Low-code When an AI system is used to accelerate the code development but a human may be involved in the writing of some code
Four colleagues mid discussion around a table strewn with printed reports and marked up pages, one of them gesturing at a line that clearly matters.

Model Quality, Ethics, and Responsible AI Principles

Phase 6 · 20 terms

Before deploying a model, understand the quality and ethical dimensions — bias, fairness, explainability, and the human oversight needed to govern outputs responsibly.

  1. 71 Model bias An error that can be introduced when approximating a real-world problem with a simplified model or a dataset that does n...
  2. 72 Model validation The process of testing a trained AI model on previously unseen data to evaluate how well it generalizes (accuracy, fairn...
  3. 73 Model testing A process for ensuring the most appropriate model is used for each task, the program handles the input correctly, and th...
  4. 74 Model access The ability to interact, use, or control an AI model
  5. 75 Model control The security controls applied to an AI model to manage its behavior, security, and compliance
  6. 76 Public model AI models accessed as cloud services that are shared among multiple customers
  7. 77 Private model AI models deployed in a logically isolated environment
  8. 78 Sanctioned AI AI tools and workflows that have been vetted and approved by an organization and meet security and privacy requirements
  9. 79 Unsanctioned AI Any AI tools that are adopted by an individual independently without organizational approval
  10. 80 Human-in-the-loop Allowing humans to interview and approve or override decisions in AI systems.
  11. 81 Overreliance When human practitioners lose critical thinking capabilities and become overly dependent on AI recommendations
  12. 82 Explainability An AI principle of the extent to which stakeholders can understand the reasoning behind a system's output
  13. 83 Transparency Everyone in the organization has access to the governance policies and understands why they are in place.
  14. 84 Fairness An AI principle that means outcomes should not disadvantage individuals based on protected or irrelevant characteristics
  15. 85 Inclusiveness An AI principle of designing, developing, and deploying AI systems that serve diverse populations equitably, ensuring ac...
  16. 86 Consistency An AI principle that AI system operates reliably in real-world applications, minimizing erratic behavior and maintaining...
  17. 87 Social impact An AI principle that AI systems should be designed, developed, and used in ways that produce beneficial effects for indi...
  18. 88 Sustainability An AI principle that examines how efficiently resources are used and the broader effects of those usages
  19. 89 Accountability Data governance plans are being followed and there are accountability measures in place
  20. 90 Accountability-AI An AI principle of who is responsible for decisions and the AI systems that support them at an organizational and indivi...
An industrial control panel up close: a physical safety interlock, a limit dial and a guarded switch under a clear cover, the metal wearing its years honestly.

AI-Specific Runtime Controls: Guardrails, Limits, and Enforcement

Phase 7 · 16 terms

Responsible AI principles must be enforced mechanically — learn the runtime controls that constrain model inputs, outputs, and resource consumption.

  1. 91 Guardrail The controls in AI systems that enforce ethical, safe, and responsible usage of AI.
  2. 92 Guardrails Software development tools that ensure automation and scripting comply with established policies, standards, and best pr...
  3. 93 Technical guardrail A control that enforces compliance with AI governance in day-to-day operations
  4. 94 Guardrail testing The process of systematically testing an AI system against safety, security, and policy rules (guardrails) to ensure it...
  5. 95 Input validation Any technique used to ensure that the data entered into a field or variable in an application is handled appropriately b...
  6. 96 Input quota The maximum amount of data or the maximum number of requests a user or application can send to an AI system within a tim...
  7. 97 Modality limit Controlling which type of input and output a model can handle
  8. 98 Token limit The maximum number of tokens an AI model can process in a single request
  9. 99 Quantity quota The maximum number of files, documents, or data items that a user or application can upload or process within a specifie...
  10. 100 Rate limitation The maximum number of requests that can be made to an AI model within a given time window
  11. 101 Rate limiting An approach that protects the attack from consuming all available bandwidth and impacting other servers and services on...
  12. 102 Throttling Technology that allows the CPU to slow down if thermal output reaches a critical level or to improve power performance.
  13. 103 Gateway control The controls between the user interface and the AI models used to enforce usage rules, prevent abuse, and ensure complia...
  14. 104 Prompt firewall A security layer within an AI gateway that inspects and filters inputs to and the outputs generated by the model
  15. 105 Counterfactual testing A method of evaluating an AI model by changing one or more input features in a small, targeted way and checking how the...
  16. 106 False positive In security scanning, a case that is reported when it should not be.
A tall stack of tabbed compliance binders on a boardroom table beside an open audit checklist ticked in pen, reading glasses set down mid page in low window light.

Governance, Risk, and Compliance Frameworks

Phase 8 · 19 terms

Individual controls need a governance envelope — introduce the risk and compliance structures, standards bodies, and regulations that frame AI accountability at scale.

  1. 107 Risk Likelihood and impact (or consequence) of a threat actor exercising a vulnerability.
  2. 108 CIA Triad Three principles of security control and management. Also known as the information security triad. Also referred to in r...
  3. 109 Due diligence A legal principle that a subject has used best practice or reasonable care when setting up, configuring, and maintaining...
  4. 110 Cost-benefit analysis A decision-making tool used in projects and procurement that compares the cost of a solution against the financial benef...
  5. 111 AI policy A written document that states what is allowed, what evidence is needed (to support audits), and who is authorized to ma...
  6. 112 AI procedure Who does what, when, and with what evidence across the AI lifecycle to put the AI policy into practice
  7. 113 Governance, Risk, and Compliance An integrated approach that aligns an organization's direction and accountability (governance), the identification and t...
  8. 114 Service level agreement Operating procedures and standards for a service contract.
  9. 115 EU AI Act A law that applies to all European Union Member States and establishes a risk-based framework; the law imposes strict co...
  10. 116 General Data Protection Regulation Provisions and requirements protecting the personal data of European Union (EU) citizens. Transfers of personal data out...
  11. 117 Payment Card Industry Data Security Standard The information security standard for organizations that process credit or bank card payments.
  12. 118 System and Organization Controls 2 An independent audit standard that evaluates how well a service organization designs and operates controls to protect cu...
  13. 119 ISO/IEC 42001:2023 An ISO standard that specifies the requirements for an AI Management System (AIMS) covering policy, roles, risk manageme...
  14. 120 NIST AI Risk Management Framework (AI RMF) A voluntary, structured guideline created to help organizations understand, evaluate, and manage risks associated with a...
  15. 121 National Institute of Standards and Technology A nonregulatory government agency that develops computer security standards used by US federal agencies and publishes be...
  16. 122 Organisation for Economic Co-operation and Development An international organization that develops and promotes common principles, guidelines, and policy standards for trustwo...
  17. 123 Open Web Application Security Project A charity and community publishing a number of secure application development resources.
  18. 124 Third-party compliance evaluation Reviews by an independent organization that assess whether an organization's systems, processes, and models meet specifi...
  19. 125 Policy as code The policy is defined, enforced, and managed using programming code rather than manual procedures
A security team gathered around one desk mid handover, someone pointing at a laptop while a glass wall of sticky notes keeps score behind them.

Roles, Teams, and Organisational Accountability

Phase 9 · 11 terms

Governance frameworks are executed by people — introduce the human roles and organisational structures responsible for building, operating, and auditing AI systems.

  1. 126 AI architect A role that outlines how AI components fit together within an organization, choosing patterns for data flows, model serv...
  2. 127 AI auditor A role that provides independent assurance and verification that approvals, monitoring, and incident handling adhere to...
  3. 128 AI governance engineer A role that integrates policy into daily operations by codifying approval workflows, managing deployment risks, automati...
  4. 129 AI risk analyst A role that assesses use cases and systems based on legal, ethical, and business risk criteria, recommends mitigations,...
  5. 130 AI Center of Excellence A cross-functional team in an organization that serves as the hub for safe and effective AI implementation and support
  6. 131 Machine learning engineer A role that takes prototypes and transforms them into production-grade models by optimizing the code, packaging dependen...
  7. 132 MLOps engineer A role that automates the lifecycle on the platform by establishing continuous integration and delivery for AI models an...
  8. 133 Platform engineer A role that provides the shared infrastructure that teams use to experiment and deploy safely
  9. 134 DevSecOps A combination of software development, security operations, and systems operations, and refers to the practice of integr...
  10. 135 Security operations center The location where security professionals monitor and protect critical information assets in an organization.
  11. 136 Accounting Tracking authorized usage of a resource or use of rights by a subject and alerting when unauthorized use is detected or...
A hand holds a hardware security key up to a badge reader beside a door, close enough to catch the reader's small green light.

Identity, Access, and Cryptographic Foundations

Phase 10 · 20 terms

Before examining threats to systems, establish the foundational security primitives — identity, authentication, authorisation, and the cryptography underpinning them.

  1. 137 Identification The process by which a user account (and its credentials) is issued to the correct person. Sometimes referred to as enro...
  2. 138 Authentication A method of validating a particular entity's or individual's unique credentials.
  3. 139 Authorization The process of determining what rights and privileges a particular entity has.
  4. 140 Multifactor authentication An authentication scheme that requires the user to present at least two different factors as credentials; for example, s...
  5. 141 Context-aware authentication An access control scheme that verifies an object's identity based on various environmental factors, like time, location,...
  6. 142 Access control The process of determining and assigning privileges to resources, objects, and data. Each resource has an access control...
  7. 143 Least privilege A basic principle of security stating that something should be allocated the minimum necessary rights, privileges, or in...
  8. 144 Role-based access control An access control model where resources are protected by ACLs that are managed by administrators and that provide user p...
  9. 145 Attribute-based access control An access control technique that evaluates a set of attributes that each subject possesses to determine if access should...
  10. 146 Identity and access management A security process that provides identification, authentication, and authorization mechanisms for users, computers, and...
  11. 147 Open authorization A standard for federated identity management, allowing resource servers or consumer sites to work with user accounts cre...
  12. 148 Encryption Scrambling the characters used in a message so that the message can be seen but not understood or modified unless it can...
  13. 149 Transport Layer Security A security protocol that uses certificates for authentication and encryption that provides end-to-end security to protec...
  14. 150 HyperText Transfer Protocol/HTTP Secure Application protocol used to provide web content to browsers. HTTP uses port 80. HTTPS(ecure) provides for encrypted tra...
  15. 151 Hardware security module An appliance for generating and storing cryptographic keys. This sort of solution may be less susceptible to tampering a...
  16. 152 Key management system In PKI, procedures and tools that centralizes generation and storage of cryptographic keys.
  17. 153 Homomorphic encryption Method that allows computation of certain fields in a dataset without decrypting it.
  18. 154 Trusted execution environment A secure area of a processor that runs code and handles data in isolation from the rest of the system
  19. 155 Trusted platform module Specification for secure hardware-based storage of encryption keys, hashed passwords, and other user- and platform-ident...
  20. 156 Fingerprint Mobile-device bio-gesture authentication mechanism that requires the user to scan his or her fingerprint to unlock the d...
A lone analyst at a darkened desk, face washed red by a screen full of intrusion alerts, the rest of the office long since gone home.

Threat Landscape: Attack Vectors and Adversarial Techniques

Phase 11 · 34 terms

With defences named, learn what they must defend against — the full catalogue of attack techniques targeting AI systems, their inputs, outputs, training pipelines, and supply chains.

  1. 157 Advanced persistent threat An attacker's ability to obtain, maintain, and diversify access to network systems using exploits and malware.
  2. 158 Malware Software that serves a malicious purpose, typically installed without the user's consent (or knowledge).
  3. 159 Social engineering Using persuasion, manipulation, or intimidation to make the victim violate a security policy. The goal of social enginee...
  4. 160 Impersonation Social engineering attack where an attacker pretends to be someone they are not.
  5. 161 Brute force attack A type of password attack where an attacker uses an application to exhaustively try every possible alphanumeric combinat...
  6. 162 Reconnaissance The actions taken to gather information about an individual or organization's computer systems and software.
  7. 163 Payload The code implemented within the message body that carries the actual intended content or action
  8. 164 Obfuscation A technique that essentially 'hides' or 'camouflages' code or other information so that it is harder to read by unauthor...
  9. 165 Reverse engineering The process of analyzing the structure of hardware or software to reveal more about how it functions.
  10. 166 Denial of service The preventing clients from connecting to a legitimate access point
  11. 167 Denial of service attack Any type of physical, application, or network attack that affects the availability of a managed resource.
  12. 168 Distributed denial of service attack An attack that uses multiple compromised hosts (a botnet) to overwhelm a service with requests or response traffic.
  13. 169 Spoofing, tampering, repudiation, information disclosure, DoS, and elevation of privilege Threat model developed by Microsoft.
  14. 170 Tactics, techniques, and procedures Analysis of historical cyberattacks and adversary actions.
  15. 171 Structured query language injection An attack that injects a database query into the input data directed at a server by accessing the client side of the app...
  16. 172 Prompt-injection attack Manipulating inputs sent to LLM to change the behavior of the LLM or cause it to leak information
  17. 173 Jailbreak Removes the protective seal and any OS-specific restrictions to give users greater control over the device.
  18. 174 Input manipulation When an attacker crafts a malicious input to exploit the model's behavior
  19. 175 Improper output handling When generated responses are used directly in applications without verification
  20. 176 Output Integrity Attack A security attack where the attacker changes or interferes with the final output of a machine learning model
  21. 177 Data poisoning When misinformation or malicious data is used in the training dataset, causing bias and changes in behavior
  22. 178 Model poisoning A security attack where harmful data or code is intentionally added during the model's training or fine-tuning process
  23. 179 Backdoor attack A cyberattack where an attacker exploits or installs a hidden method of bypassing normal authentication or security cont...
  24. 180 Trojan attack A cyberattack that uses a malicious program disguised as legitimate software to trick users into installing it, after wh...
  25. 181 Membership inference A security attack where an attacker gains access for sensitive information when the history is saved and used for traini...
  26. 182 Model inversion attack A security attack where an attacker trains the model with a specific set of records and uses it to inquire the data used...
  27. 183 Model theft A security attack where an attacker reverse engineers the organization's machine learning model to gain access to traini...
  28. 184 Transfer learning attack A security attack where an attacker exploits pre-trained models with inherited vulnerabilities utilized for fine-tuning
  29. 185 AI Supply Chain Attack A security attack where an attacker compromises one or more components used to build, train, deploy, or operate an AI sy...
  30. 186 Deep fake The use of artificial intelligence and machine learning to generate a highly-realistic video of a person. A fake video r...
  31. 187 Disinformation A type of attack that falsifies an information resource that is normally trusted by others.
  32. 188 Misinformation Repeating false claims or rumors without the intention to deceive
  33. 189 Sensitive information disclosure When the model unintentionally reveals private or proprietary data
  34. 190 Data breach When confidential or private data is read, copied, or changed without authorization. Data breach events may have notific...
Two developers share one desk and one mechanical keyboard, a code review diff open on screen with comments already stacking up beside the notes.

Defensive Technologies and Secure Development Practices

Phase 12 · 21 terms

Map defences directly to the attacks just catalogued — the technical controls, secure coding practices, and protective tools that harden AI and traditional systems alike.

  1. 191 Secure Software Development Life Cycle A method of system development that incorporates security controls in every phase of the system's lifecycle.
  2. 192 Infrastructure as code Provisioning architecture in which deployment of resources is performed by scripted automation and orchestration.
  3. 193 Static application security testing Testing that is done early in the software development life cycle to examine the code for security vulnerabilities.
  4. 194 Dynamic application security testing Testing that is done after code is placed into production and is able to unearth vulnerabilities that are evident once t...
  5. 195 Software composition analysis Tools designed to assist with identification of third-party and open-source code during software development and deploym...
  6. 196 Software bill of materials Inventory of third-party and open-source code components used in an application or package.
  7. 197 Code linting A process where an AI tool provides recommendations for errors, bugs, and inconsistencies within code
  8. 198 Code quality tool A software tool that automatically analyzes source code used to build or integrate AI systems to detect bugs, security v...
  9. 199 Unit test The developer writes a simple 'pass/no pass' test for code. This ensures that a particular block of code performs the ex...
  10. 200 Integrated development environment A software application that provides comprehensive tools and features to facilitate computer programming and software de...
  11. 201 Command-line interface A text-based interface between the user and the operating system that accepts input in the form of commands.
  12. 202 Data anonymization The process of removing personally identifiable information from data sets, so that the individuals the data describes r...
  13. 203 Data masking A de-identification method where generic or placeholder labels are substituted for real data while preserving the struct...
  14. 204 Data redaction The process of removing, obscuring, or masking data that is not necessary after collection
  15. 205 Data loss/leak prevention A software solution that detects and prevents sensitive information from being stored on unauthorized systems or transmi...
  16. 206 Log scrubbing A process that removes, hides, or encrypts confidential data in log files.
  17. 207 Watermark Information displayed as an overlay on a report indicating content is confidential or should not be printed or distribut...
  18. 208 Watermarking In data protection, methods and technologies that apply a unique anti-tamper signature or message to a copy of a documen...
  19. 209 Cryptographic watermark Hidden markers or codes added to the model's output that are difficult to forge or remove
  20. 210 Fingerprint_ai Hidden identifiers that are embedded into content, that assist in detecting unauthorized use/copying and provide a trail...
  21. 211 Prompt monitoring The ongoing tracking and analysis of the prompts (inputs) sent to an AI system—along with the model's responses—to detec...
A security operations centre seen from the back of the room, a wall of dashboards and a world map glowing over two analysts at their desks.

Detection, Monitoring, and Threat Intelligence

Phase 13 · 10 terms

Defences eventually fail — learn how to detect, correlate, and investigate anomalies in real time across both AI-specific and traditional security telemetry.

  1. 212 Anomaly detection The process of evaluating traffic flows and communications across the network and comparing the current activity levels...
  2. 213 Pattern detection The process of automatically finding recurring structures, trends, or regularities in data (such as similarities, sequen...
  3. 214 Signature matching A technique in which predefined patterns are used to identify malicious activity.
  4. 215 Intrusion detection system A security appliance or software that analyzes data from a packet sniffer to identify traffic that violates policies or...
  5. 216 Log monitoring The continuous collection, review, and analysis of system and application logs that record an AI model's inputs, outputs...
  6. 217 Automated data correlation The process where an AI system automatically detects and links relationships, patterns, or associations between differen...
  7. 218 User and entity behavior analytics A system that can provide automated identification of suspicious activity by user accounts and computer hosts.
  8. 219 Endpoint detection and response A software agent that collects system data and logs for analysis by a monitoring system to provide early detection of th...
  9. 220 Security information and event management A solution that provides real-time or near-real-time analysis of security alerts generated by network hardware and appli...
  10. 221 Performance monitoring Tracking the number of requests and responses being submitted and supplied by the system to improve the model
An incident war room at night: five people around a table of open laptops, a whiteboard timeline behind them, and enough takeaway cups to date the outage.

Incident Response, Evaluation, and Knowledge Bases

Phase 14 · 16 terms

When detection fires, teams need structured response plans, scoring frameworks, and curated knowledge bases to triage, measure, and learn from AI security events.

  1. 222 Incident response plan Specific procedures that must be performed if a certain type of event is detected or reported.
  2. 223 Playbook A checklist of actions to perform to detect and respond to a specific type of incident.
  3. 224 Security orchestration, automation, and response A class of security tools that facilitates incident response, threat hunting, and security configuration by orchestratin...
  4. 225 AI Incident Database A collection of AI incidents reported in real time
  5. 226 AI Vulnerability Database An open-source knowledge base that collects the data about failures modes for AI models, datasets, and systems
  6. 227 AI threat modeling The process of identifying potential threats and analyzing the risks in AI systems
  7. 228 Common Vulnerabilities and Exposures A scheme for identifying vulnerabilities developed by MITRE and adopted by NIST.
  8. 229 Common Weakness Enumeration A community-developed list and classification system for common software and hardware security flaws (weaknesses) that c...
  9. 230 MITRE Adversarial Threat Landscape for Artificial-Intelligence Systems A globally accessible framework developed to understand the various techniques and tactics adversaries use to attack AI...
  10. 231 MITRE Common Vulnerability and Exposures (CVE) Database A publicly available catalog of known cybersecurity vulnerabilities, where each vulnerability is assigned a unique CVE I...
  11. 232 NIST National Vulnerability Database (NVD) A US government-maintained, publicly accessible repository of standardized information about known cybersecurity vulnera...
  12. 233 Massachusetts Institute of Technology AI Risk Repository A collection of AI risks classified by frameworks and seven domains
  13. 234 ArXiV An open-source platform where researchers share scientific papers
  14. 235 Honeypot A host (honeypot), network (honeynet), file (honeyfile), or credential/token (honeytoken) set up with the purpose of lur...
  15. 236 Bilingual evaluation understudy An automatic metric for evaluating machine-translated text by comparing it to one or more human-created reference transl...
  16. 237 Recall-oriented understudy for gisting evaluations A set of metrics for evaluating how good a system-generated summary or text is by comparing it to one or more human (ref...
A technician walks the cold aisle of a data centre with a tablet, rack doors open on one side and rows of status lights receding into the dark.

MLOps, Continuous Delivery, and Operational Resilience

Phase 15 · 8 terms

Close the loop — operational practices, drift management, and delivery discipline that keep deployed AI systems healthy, current, and continuously improving.

  1. 238 Continuous integration/continuous deployment A software development method combining app and platform updates (rapidly committed to production) with code updates (ra...
  2. 239 Change control system A collection of formal, documented procedures for changing official project documents and how project deliverables will...
  3. 240 Rollback To restore a system to the last known good state.
  4. 241 Regression testing The process of testing an application after changes are made to see if these changes have triggered problems in older ar...
  5. 242 Data drift A phenomenon where the statistical properties of input data change over time, potentially reducing the accuracy and reli...
  6. 243 Model drift A phenomenon wherein the statistical properties of the target variable or features of a machine learning model change ov...
  7. 244 Model skewing When the behavior of a machine learning model is altered through biased or manipulated training or feedback data
  8. 245 Key performance indicator Metrics generally used to evaluate factors crucial to the success of a project or organization. In project management, a...