2
$\begingroup$

I've been working on neural network for a while and I built simple network from scratch with python but before using TensorFlow, I would like to have a complete understanding of it.

Here is my question :

Lets say you have 3 layers you have 3 weights to update :

1) --> the weight between the outputlayer and the hiddenLayer2

2) --> the weight between the hiddenLayer2 and the hiddenLayer1

3) --> the weight between the hiddenLayer1 and the inputLayer

For the 1) the calculation is quite simple we got :

weight_3 += LEARNING_RATE * ((2*(target - output)) * sigmoid'(output) * layer2)

For the 2) the calculation is more complicated and we got :

weight_2 += LEARNING_RATE * ((2*(target - output)) * sigmoid'(output) * weight_3) * sigmoid'(hiddenLayer2)

I need help for the 3rd part, I tried to calculate and find on internet but not a lot of people uses 2 hidden layer when they work from scratch.

I also tried to resolve the chain rule but its too long and I can't resolve.

Does someone know the formula to get the weight between the hiddenLayer1 and the inputLayer ?

Thank you so much in advance

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Here is a good example of how to implement forward and backward propagation, in numpy, but there should be similar functions in tf.

Improve this answer
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.