Credits

1. “Neural Networks and Deep Learning” by Michael A. Nielsen

This textbook wouldn’t exist without Michael Nielsen. His textbook was fantastic, and I wanted to create something similar to it in an effort to practice my writing and teaching skills. Truthfully, I’d still recommend his textbook over my own unless you’re strapped for time.

2. “Neural Networks” by 3Blue1Brown

3Blue1Brown’s teaching style emphasizes understanding, and this textbook heavily leaned (or attempted to) on that style. His resource on neural networks was fantastic, and this textbook attempts to provide the more mathematical side of them.

3. “How to Create a Neural Network (and Train it to Identify Doodles)” by Sebastian Lague

Many of the widgets used are based on the demonstrations shown in Sebastian Lague’s video. This includes the widgets where you tune a neural network, and the ones where you can watch a neural network attempt to classify points on a graph.

4. “What is Automatic Differentiation?”

Provided a useful comparison of numerical and symbolic differentiation to automatic differentiation that was used in an attempt to inspire backpropagation.

5. “Chain Rule + Dynamic Programming = Neural Networks”

Dynamic programming isn’t talked about much when discussing backpropagation, and this blog made me aware of the connection between the two while doing research on backpropagation.