Self-Attention Mechanism
Transformer Neural Networks
Causal Breakdown & Key Mechanics
The Context Problem: Isolated dictionary lookups fail because words like 'bank' or 'lead' shift meaning depending on neighbors.
The Triplet Roles: Each word creates a Query ('What am I looking for?'), a Key ('What am I about?'), and a Value ('What information do I carry?').
The Matching Score: When Word A's Query matches Word B's Key, they produce a high attention score.
The Weighted Blend: The word updates its meaning by taking a weighted sum of the Values of its most relevant neighbors.
Feynman Anchor Metaphors
Ground abstract mechanisms in tangible everyday physical intuition
The Cocktail Party & Clipboards
Imagine a crowded party where everyone holds a clipboard with questions (Query) and name tags (Key). When you shout your question, people with matching tags wave back, and you listen to their stories (Value) while ignoring the rest.
Echo Socratic Mentor
Grounded strictly in your course material