I am working on a image classification problem using Transfer Learning.
Parameters used given below:
Adam optimizer with learning rate - 0.0001
adamOpti = Adam(lr = 0.0001)
model.compile(optimizer = adamOpti, loss = "categorical_crossentropy, metrics = ["accuracy"])
For testing I used adam
optimizer without explicitly specifying any parameter (default value lr = 0.001).
With the default value of learning rate the accuracy of training and validation got stuck at around 50%.
And when I use learning rate = 0.0001 in the first epoch itself I could see that the accuracy is going to 90%.
Could you please help me understand
1. why with lower value of learning rate the accuracy is increasing rapidly?
2. Also which of the above used learning rate is the better?
3. How could I make use of decaying learning rate in Keras?
Thank you