A deeper pass through the MLP training dynamics from karpathy-03-mlp: initial loss calibration, saturated tanh activations, gradient flow, and principled weight initialisation. These notebooks inspect what the network is doing internally at startup, not just whether the final loss goes down.
2 notebooks: poor network initialisation
Sources: Andrej Karpathy - Zero to Hero · makemore