Introduction to Transformers & Attention
Transformers underpin modern NLP and vision, and attention is the key mechanism to understand.
Learn it
Read Jay Alammar's 'The Illustrated Transformer' for a step-by-step visual walkthrough of self-attention, multi-head attention, and positional encoding. Then watch 3Blue1Brown's attention chapter to see query/key/value dot-products and softmax update embeddings with context.
By the end you’ll be able to
- Explain self-attention with queries, keys, and values
- Describe why transformers replaced RNNs for sequences
Free resources
Practice
Hand-compute a tiny single-head attention: given three toy word vectors, form Q/K/V, compute attention scores, softmax them, and produce one output vector.
Build it — your output
A worked example computing one attention step end-to-end with numbers.
Prove it — check yourself
You’re ready to move on when you can honestly say yes to each:
- What do the query, key, and value vectors each represent?
- Why is attention parallelizable where RNNs are not?
- Why are positional encodings needed?
Do it with AI
Use a free AI tool as your tutor and pair-programmer. Copy a prompt below into Claude, ChatGPT, or Gemini (for building, try Cursor or Copilot).
The habit: Try it yourself first — a little struggle is where the learning actually happens. Then ask for a hint before the answer — never copy blindly.
- Understand itClaude / ChatGPT / Gemini
I'm a beginner learning "Introduction to Transformers & Attention". Explain it in simple terms with a real-world analogy, then ask me ONE question to check my understanding — don't reveal the answer.
- PracticeClaude / ChatGPT / Gemini
Give me 3 short practice problems on "Introduction to Transformers & Attention" (my goal: explain self-attention with queries, keys, and values), from easy to hard. Don't include the solutions yet — I'll try them first, then ask you to check.
- Get unstuckClaude / ChatGPT / Gemini
I'm stuck on this exercise about "Introduction to Transformers & Attention": [paste your attempt here]. Give me ONE small hint to move forward — do NOT give me the full solution.
- Build & reviewCursor / Copilot
Here is my work for a "Introduction to Transformers & Attention" task: [paste your code or answer]. Review it: point out what's wrong and WHY, and suggest improvements — but let me make the fixes myself.
- Prove itClaude / ChatGPT / Gemini
Quiz me on "Introduction to Transformers & Attention". Ask me 5 questions one at a time, wait for my answer each time, then tell me if I'm right and briefly why.