Overfitting & Regularization
Models that ace training data but fail on new data are the #1 ML trap. Understanding this separates beginners from practitioners.
Learn it
Learn overfitting vs underfitting (the bias–variance tradeoff), how to detect it (train vs test gap), and how regularization and simpler models help.
By the end you’ll be able to
- Recognise overfitting
- Apply regularization
Free resources
- StatQuest — Bias-Variance Trade-Off (video)(opens in a new tab)Start here
The core intuition behind overfitting
Free~6 min - Kaggle Learn — Underfitting and Overfitting(opens in a new tab)
Detect over/underfitting in a real model
Free3 hr - scikit-learn — Linear Models (Ridge & Lasso)(opens in a new tab)
L1/L2 regularization reference
Free
Practice
Deliberately overfit a model (too complex), then show test error rising, then fix it.
Build it — your output
A notebook demonstrating overfitting and a fix (regularization or simpler model), with train/test curves.
Prove it — check yourself
You’re ready to move on when you can honestly say yes to each:
- Can you spot overfitting from train/test scores?
- Do you understand the bias–variance tradeoff?
- Can you apply a regularization technique?
Do it with AI
Use a free AI tool as your tutor and pair-programmer. Copy a prompt below into Claude, ChatGPT, or Gemini (for building, try Cursor or Copilot).
The habit: Try it yourself first — a little struggle is where the learning actually happens. Then ask for a hint before the answer — never copy blindly.
- Understand itClaude / ChatGPT / Gemini
I'm a beginner learning "Overfitting & Regularization". Explain it in simple terms with a real-world analogy, then ask me ONE question to check my understanding — don't reveal the answer.
- PracticeClaude / ChatGPT / Gemini
Give me 3 short practice problems on "Overfitting & Regularization" (my goal: recognise overfitting), from easy to hard. Don't include the solutions yet — I'll try them first, then ask you to check.
- Get unstuckClaude / ChatGPT / Gemini
I'm stuck on this exercise about "Overfitting & Regularization": [paste your attempt here]. Give me ONE small hint to move forward — do NOT give me the full solution.
- Build & reviewCursor / Copilot
Here is my work for a "Overfitting & Regularization" task: [paste your code or answer]. Review it: point out what's wrong and WHY, and suggest improvements — but let me make the fixes myself.
- Prove itClaude / ChatGPT / Gemini
Quiz me on "Overfitting & Regularization". Ask me 5 questions one at a time, wait for my answer each time, then tell me if I'm right and briefly why.