Hands-on builds with annotated notebooks.
Makemore — Andrej Karpathy
9 notebooks
Backprop & Autograd
Scalar autograd engine, manual backprop, automated backward pass, and a minimal neural net library.
5 notebooks + 2 helpers
Bigram Language Model
Character-level counting model, sampling, loss, smoothing, and an equivalent neural net.
5 notebooks
MLP / Makemore
Scaling the bigram model to an MLP with embeddings, train/val/test splits, and experiments.
1 notebook
BatchNorm
Training dynamics, activation statistics, gradient flow, initialization, and normalization.
Algorithms