The training loop for a small, from-scratch decoder-only transformer — four steps, start to finish. Click any box to see its code. Click any highlighted word to jump to where it's defined, as deep as you want to go.
The complete files behind every box above. Words are clickable here too.