Timeline for Backpropagation Mathematics with Sigmoid Output Activation and Cross Entropy Loss
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 10, 2021 at 19:52 | history | edited | Brian Spiering |
edited tags
|
|
| Oct 12, 2020 at 17:53 | comment | added | Coldchain9 | Great! Thank you. I actually have this exact formulation derived out as well (with the transpose) so we were definitely on the same page in regards to matrix version. Thank you again for your help. | |
| Oct 12, 2020 at 17:04 | comment | added | Javier TG | All right. Then if in the future $W^{[2]}$ is a matrix be careful, because then you would need to express $\partial C/\partial W^{[2]}$ as: $$ \frac{\partial C}{\partial W^{[2]}} = (\sigma - y)(A^{[1]})^T$$ | |
| Oct 12, 2020 at 16:37 | comment | added | Coldchain9 | Great! With my single training example with say 4 hidden layer nodes my W[2] is 1x4 (row vector) and A[1] is 4x1 (column vector). I just used the matrix terminology so I could expand this idea to batch training in the future. Thank you for checking my work. | |
| Oct 12, 2020 at 14:12 | comment | added | Javier TG | Yes, that's right. Just a thing to care about (assuming this is for a fully-connected NN) is that, in your case, both $A^{[1]}$ and $W^{[2]}$ are vectors. | |
| Oct 12, 2020 at 12:34 | history | asked | Coldchain9 | CC BY-SA 4.0 |