Academy K

Backpropagation Through a Network

InterviewDerivationbackpropagationchain rulegradients

Related module: Learning from data and backpropagation

Problem statement

Consider a two-layer neural network:

h=σ(W1x+b1)h = \sigma(W_1 x + b_1)

y=W2h+b2y = W_2 h + b_2

L=12∥y−t∥2L = \frac{1}{2}\|y - t\|^2

where σ\sigma is an element-wise activation and tt is the target.

Derive the gradients ∂L∂W2\frac{\partial L}{\partial W_2}, ∂L∂W1\frac{\partial L}{\partial W_1}, ∂L∂b2\frac{\partial L}{\partial b_2}, and ∂L∂b1\frac{\partial L}{\partial b_1}.

Hints

Practise this problem

Verify you understand it—not just that you read the answer.

Learn the foundations

The AI / SI track covers this topic and more in depth.