Introduction to Neural Networks

This course is meant to be a lightweight introduction to neural networks, so that you can begin to understand them. Current large language models use these same ideas and more, and we’ll explore those in later courses. There are many different paradigms within neural networks, so the focus is on preparing you to be able to explore the field as you wish.

This course is going to focus on what are known as feed-forward networks. In practice, they give us extremely promising results. In fact, the following widget uses them to, with decent accuracy, identify what numbers are drawn:

Course Structure

This course introduces the fundamentals of neural networks, including how they’re designed, how they’re trained, and what the algorithms behind them are actually doing. You’ll learn the core concepts and build a complete neural network from scratch that can recognize handwritten digits.

This course is lab-heavy. You will be implementing an entire neural network from scratch using Python and NumPy. That means no help from higher-level libraries. Although such libraries are helpful for speed and efficiency, they abstract away what’s actually going on in a neural network, which isn’t helpful in developing your understanding.

Prerequisites

The required mathematical background you’ll need is fairly straightforward. If you’re comfortable with basic algebra, you’ll be in good shape for most of the material. For the labs, we’ll be implementing everything in Python, using the NumPy library for numerical work, so some programming experience is assumed. Some linear algebra—such as vectors, matrices, matrix multiplication, dot products, and transposing—will come up regularly. Calculus appears as well, mainly in the form of derivatives and gradients, but I’ve done my best to keep things approachable. Probability does show up in a couple of places, so it can be helpful to know, but you’re able to skip over those parts without missing much. If you’d like to brush up beforehand or want a refresher, check out our Mathematics roadmap to find resources and track what you already know.

Looking Forward

We’re going to start exploring neural networks with the goal of creating a neural network that can detect and classify digits correctly. To do this, we’re going to dive into the architecture of neural networks and how to train them.