I've been working on neural network for a while and I built simple network from scratch with python but before using TensorFlow, I would like to have a complete understanding of it.
Here is my question :
Lets say you have 3 layers you have 3 weights to update :
1) --> the weight between the outputlayer and the hiddenLayer2
2) --> the weight between the hiddenLayer2 and the hiddenLayer1
3) --> the weight between the hiddenLayer1 and the inputLayer
For the 1) the calculation is quite simple we got :
weight_3 += LEARNING_RATE * ((2*(target - output)) * sigmoid'(output) * layer2)
For the 2) the calculation is more complicated and we got :
weight_2 += LEARNING_RATE * ((2*(target - output)) * sigmoid'(output) * weight_3) * sigmoid'(hiddenLayer2)
I need help for the 3rd part, I tried to calculate and find on internet but not a lot of people uses 2 hidden layer when they work from scratch.
I also tried to resolve the chain rule but its too long and I can't resolve.
Does someone know the formula to get the weight between the hiddenLayer1 and the inputLayer ?
Thank you so much in advance