# Help Using Backprop Formulas from Deep Learning Book

I am trying to see how the fundamental formulas of backpropagation from this online book work on a small example.

The author lists these 4 fundamental equations (where I believe from the reading that the cost (or loss) function C as well as the derivative of the activation function $\sigma$ are general and work with any cost or activation function): If I take a small example network: and assume....

• C = $\frac{1}{2}(a^3-y)^2$ (squared error)
• The activation (a) for each node is just the logistic function sigmoid

From here we have the following:

• $z^2_1=w^1_{11}x_1+w^1_{12}x_2+b_1$
• $z^2_2=w^1_{21}x_1+w^1_{22}x_2+b_2$
• $z^2_3=w^1_{31}x_1+w^1_{32}x_2+b_3$
• $a^2_1 =\sigma(z^2_1)$
• $a^2_2 =\sigma(z^2_2)$
• $a^2_3 =\sigma(z^2_3)$
• $z^3_1=w^2_{11}a^2_1+w^2_{12}a^2_2+w^2_{13}a^2_3$
• $a^3=\sigma(z^3_1)$

If I want to calculate $\frac{\partial C}{\partial w^2_{11}}$ it seems from the book that I should calculate $a^1_1\delta^2_1$ where $\delta^2 = ((w^{2+1})^T\delta^{2+1})\odot\sigma'(z^2)$.

Since $W^3$ does not exist I assume it is ignored and we can calculate $\delta^2_1= \delta^{2+1}_1\odot\sigma'(z^2_1)$.

$\delta^3_1 = \frac{\partial C}{\partial z^3_1} = (a^3-y)\sigma'(z^3_1)$ so $\frac{\partial C}{\partial w^2_{11}}= a^1_1*((a^3-y)\sigma'(z^3_1))*\sigma'(z^2_1)$. $a^1_1$ doesnt exist except if this would mean $x_1?$

However, if I just calculate $\frac{\partial C}{\partial w^2_{11}}$ I get $\frac{\partial }{\partial w^2_{11}}\frac{1}{2}([\sigma(w^2_{11}a^2_1+w^2_{12}a^2_2+w^2_{13}a^2_3)]-y)^2=a^2_1(a^3_1-y)\sigma'(z^3_1)$

Where am I going wrong?

My mistake was silly. I did not notice in the book that the weight matrices are numbered as the layer they are going into..so my $w^2$ above should be $w^3$ and then the result from the book is $\frac{\partial }{\partial w^3_{11}}=a^2_1(a^3_1-y)\sigma'(z^3_1)$ exactly as calculated by just doing the derivatives.