Introduction to Neural Networks
This short textbook is meant to be a lightweight introduction to neural networks, so that you can start understanding them. Current large language models use the same ideas and more, but that’s outside the scope of this book. The reasoning is that there are many different paradigms within neural networks, so I decided to take on an approach that would let you explore the field as you wish.
This textbook is going to focus on what are known as feed-forward networks. In practice, they give us extremely promising results. In fact, the following widget uses them to, with decent accuracy, identify what numbers are drawn:
✏️ Draw a digit (0-9) in the grid above
Use your mouse or finger to draw. The neural network will recognize your handwriting in real-time.
Course Structure
This course introduces the fundamentals of neural networks, including how they’re designed, how they’re trained, and what the algorithms behind them are actually doing. You’ll learn the core concepts and build a complete neural network from scratch that can recognize handwritten digits.
This textbook is lab-heavy. You will be implementing an entire neural network from scratch using Python and NumPy. That means no help from high-level libraries such as PyTorch. Although such libraries are helpful for speed and efficiency, they abstract away what’s actually going on in a neural network, which isn’t helpful in developing an understanding.
Prerequisites
The required mathematical background you’ll need is fairly straightforward. If you’re comfortable with basic algebra, you’ll be in good shape for most of the material. For the labs, we’ll be implementing everything in Python, using the NumPy library for numerical work, so some programming experience is assumed. Some linear algebra—such as vectors, matrices, matrix multiplication, dot products, and transposing—will come up regularly. Calculus appears from time to time, mainly in the form of derivatives and gradients, but I’ve done my best to keep things approachable. Probability does show up in a couple of places, so it can be helpful to know, but you’re able to skip over those parts without missing much.
Looking Forward
We’re going to start exploring neural networks with the goal of creating a neural network that can detect and classify digits correctly. To do this, we’re going to dive into the architecture of neural networks and how to train them.