Chapter 02

Attention, per decode step

What a new token needs from the past.

Same five tokens, one forward pass. This time we watch what one attention layer does with them.

01 / 14

The math, once

For one head, with $Q = XW_Q$ , $K = XW_K$ , $V = XW_V$ :

\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\!\left(\frac{QK^{\top}}{\sqrt{d_k}}\right)V

Row $i$ of the result only uses row $i$ of $Q$ , but every row $\le i$ of $K$ and $V$ . That asymmetry is the whole next chapter.