SECAI Core · phase 2 of 15 · noun

Transformer

A neural network architecture that excels at processing and analyzing sequences of data by focusing on the connections between different pieces of information in a sequence

The Explain card

Plain English
A transformer is a neural network architecture built for sequences. It uses attention, which lets every piece of the input weigh its connection to every other piece, rather than reading strictly in order.
Example
Nearly every modern large language model is a transformer. When it reads "the key was in the safe, and it was rusty", attention links "it" back to "key".
Why it matters
Attention is why prompts can be influenced from anywhere in the context window. Instructions buried deep in a pasted document can carry as much weight as the user's actual request.
Hook
Every word looks at every other word, and so can an injected one.

Word knowledge

How the word is built, where it came from, and what it sits beside in memory.

In a sentence

In a prompt-injection evaluation, testers insert an instruction into a retrieved PDF and inspect the attention weights the transformer assigns to those tokens.

How it is built
  • trans- prefix across
  • form root shape
  • -er suffix one that does
Where it came from
Origin
Latin transformare, from trans (across) plus formare (to form), via Old French transformer; English agent noun in -er
Entered the language
c. 1600
What changed
The verb transform entered English in the late 1300s for a change of shape. Around 1600 the agent noun named a person who made that change. Electrical engineers applied it in the 1880s to the coil that steps voltage. In 2017 Vaswani and colleagues applied the same agent noun to their sequence model.
How it is spelled
Pattern
trans- keeps its s before a consonant
Pattern
final -rm does not double before -er
Pattern
agent -er attaches to the verb stem with no extra letter
Breaks the pattern
Latin and German keep -ator (transformator); English built transformer on the verb with native -er

Spelled like

  • transfer
  • transmit
  • former
  • performer
Broken into chunks
  • trans
    • transfer
    • transmit
    • transparent
  • form
    • formation
    • reform
    • conform
  • er
    • encoder
    • decoder
    • performer
What it sits beside

Same subject

Same shape

  • performer
  • informer
  • reformer

Neural network architectures

Latin trans- compounds

  • transfer
  • transmit
  • transform

Agent nouns in -er

  • encoder
  • decoder
  • tokenizer

Where it sits in the deck

Phase 2: How Models Are Built: Architectures and Learning Mechanics

With the paradigms named, zoom in on the architectural building blocks that turn data into trained artifacts.

Neighbours in this phase