Explains why training accuracy is misleading, and introduces the training, validation, and test set paradigm along with overfitting, underfitting, and cross-validation as tools for honestly measuring performance.

Covers learning curves, the bias-variance tradeoff, and metrics beyond accuracy such as precision, recall, and confusion matrices for understanding how and why a model fails.

A hands-on lab to add data splitting, learning curves, and evaluation metrics to the digit classifier, and use them to diagnose its behavior.

Examines the vanishing gradient problem and why sigmoid falls short in deep networks, then explores ReLU, Leaky ReLU, GELU, and Softmax, and how the choice of activation shapes what a network can learn.

Introduces cross-entropy and log-likelihood as principled loss functions for classification, explaining why mean squared error breaks down and how the right cost function accelerates learning.

A lab to swap in ReLU, softmax, and cross-entropy loss, then measure how each change affects training speed and accuracy.

Covers techniques for keeping values well-scaled throughout the network, from normalizing inputs and choosing principled weight initializations like Xavier/He to batch normalization, which re-centers and rescales activations to stabilize deep networks.

Explores momentum and adaptive learning rate methods such as RMSProp and Adam, which smooth out noisy updates and escape shallow local minima more effectively than vanilla SGD.

A lab to implement better initialization, batch normalization, and Adam, comparing convergence against the original training loop.

Covers L1 and L2 regularization, weight decay, and early stopping, building intuition for how penalizing large weights encourages a network to learn simpler, more generalizable solutions.

Explores dropout as implicit ensembling and data augmentation as a way to expand the training set, along with other practical techniques for closing the gap between training and validation performance.

A lab to apply weight decay, dropout, early stopping, and augmentation, using validation curves to measure their impact on generalization.

Covers practical tuning strategies including learning rate schedules, batch size, grid and random search, and experiment tracking, along with a checklist for debugging networks that fail to train.

A capstone lab combining every technique from the course into a tuned, well-validated digit classifier, with a final evaluation on the held-out test set.

Reviews the key ideas of the course and offers guidance on where to go next in your machine learning journey.

A comprehensive list of credits for the sources that inspired and informed the course material. Useful for further reading.