Introduces the course structure, learning objectives, and the prerequisites needed to get started.

Explores modern activation functions such as ReLU and Softmax, examining why they outperform the sigmoid in deep networks and how the choice of activation shapes what a network can learn.

Introduces cross-entropy and log-likelihood as principled loss functions for classification, explaining why mean squared error breaks down and how the right cost function accelerates learning.

Covers regularization techniques to combat overfitting, building intuition for how penalizing large weights encourages a network to learn simpler, more generalizable solutions.

Covers techniques for keeping values well-scaled throughout the network, from normalizing inputs and choosing principled weight initializations like Xavier/He to batch normalization, which re-centers and rescales activations throughout training to combat internal covariate shift and stabilize deep networks.

Explores momentum and adaptive learning rate methods that accelerate gradient descent, smoothing out noisy updates and escaping shallow local minima more effectively than vanilla SGD.

Introduces the training, validation, and test set paradigm and covers practical tuning strategies including early stopping, learning rate schedules, and variable learning rates.

A comprehensive list of credits for the sources that inspired and informed the course material. Useful for further reading.