I am doing Andrew Ng's excellent new Deeplearning course. I have an issue with implementing back propagation in a one hidden layer network that looks like this:

enter image description here

I am trying to derive the derivative of Z1 using the formula:

enter image description here

My understanding is that g prime Z1 in this formula is the derivative of g(z) with respect to Z which I believe is

    sigmoid(Z) * (1-sigmoid(Z))

where the function sigmoid generates the sigmoid function with respect to Z. My final equation looks like this:

  dZ1 = np.dot(W2.T, dZ2) * (sigmoid(Z1)*(1-sigmoid(Z1)))

Where Z1 is the output of the first (and only) hidden layer.

Where am I going wrong?

  • $\begingroup$ I don't see anything obviously wrong (except Z1 is not the output of the layer, it is the "logit" and A1 is the output, A1 = sigmoid(Z1)). What is happening that makes you think your code is wrong? $\endgroup$ Commented Aug 25, 2017 at 9:15
  • $\begingroup$ The course provides expected answers which up to this point, I have been getting right. However, for this section of the exercise, my result for all parameters that involve dZ1 are wrong (but the other parameters are correct). $\endgroup$
    – GhostRider
    Commented Aug 25, 2017 at 9:21
  • $\begingroup$ OK, I think I see the problem. You have made an incorrect assumption about the network architecture, take a look at the feed-forward logic again. I don't want to break the course honour code and spell it out. Note I made the same error in my previous comment, because I assumed you were correct . . . $\endgroup$ Commented Aug 25, 2017 at 9:28
  • $\begingroup$ I took the course too. The honour code suggests that I should not actually debug or write answers directly on coursework, but I think hints and general knowledge about backprop are OK. So also asking about a similar network you were implementing in a different approach or "why does it work like this" are fine. I suggest don't post your answer here because that would be publishing a partial answer to course materials. $\endgroup$ Commented Aug 25, 2017 at 9:48
  • $\begingroup$ If you have enough time to write about a different network, without using the course materials, but with the same mistake in backprop, then I think a self-answer would be fine here, and not a violation of course honour code. There are literally dozens of posts like that here, that anyone could look up. However, it may not be worth your time. $\endgroup$ Commented Aug 25, 2017 at 9:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.