SECAI Core · phase 2 of 15
Transformer
A neural network architecture that excels at processing and analyzing sequences of data by focusing on the connections between different pieces of information in a sequence
The Explain card
- Plain English
- A transformer is a neural network architecture built for sequences. It uses attention, which lets every piece of the input weigh its connection to every other piece, rather than reading strictly in order.
- Example
- Nearly every modern large language model is a transformer. When it reads "the key was in the safe, and it was rusty", attention links "it" back to "key".
- Why it matters
- Attention is why prompts can be influenced from anywhere in the context window. Instructions buried deep in a pasted document can carry as much weight as the user's actual request.
- Hook
- Every word looks at every other word, and so can an injected one.
Where it sits in the deck
Phase 2: How Models Are Built: Architectures and Learning Mechanics
With the paradigms named, zoom in on the architectural building blocks that turn data into trained artifacts.
