I have a dataset of size ~100,000 of images, I'm training a CNN model on them for regression.

  • optimizer: Adam
  • batch_size: 64
  • Number of epochs: 50

When I set the learning rate for Adam optimizer for $1e-2$ and $1e-3$, training and validation costs were normal and soon converged to good and predictable values. But now I've set the lr for $1e-1$ to just test it but training and validation costs are strange values, first $80k$ ! and after many epochs now they are reached to $600$. What's wrong with my hyperparameters, etc?

What about the decay in this situation? what should it be? its current value is $\frac{1e-1}{80}$

I really appreciate any help.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.