Transformer Self-Attention Mechanism Explained

Question: Explain the self-attention mechanism in Transformers. Key points: Self-attention lets every token "look at" every other token to build contextual representations, using three learned projections per token: Query (what am I looking for), Key (what do I contain), and Value (what information do I provide). The attention score is computed as Attention(Q, K, V) = softmax(QK^T / sqrt(d_k)) V. Multi-head attention runs several such attention computations in parallel with different learned projections so different heads can specialize (e.g. syntax vs entity relationships). Since attention has no inherent notion of order, positional encodings (e.g. sine/cosine signals) are added to token embeddings, and residual connections + layer normalization keep deep stacks trainable.