1
$\begingroup$

I have a dataset of size ~100,000 of images, I'm training a CNN model on them for regression.

  • optimizer: Adam
  • batch_size: 64
  • Number of epochs: 50

When I set the learning rate for Adam optimizer for $1e-2$ and $1e-3$, training and validation costs were normal and soon converged to good and predictable values. But now I've set the lr for $1e-1$ to just test it but training and validation costs are strange values, first $80k$ ! and after many epochs now they are reached to $600$. What's wrong with my hyperparameters, etc?

What about the decay in this situation? what should it be? its current value is $\frac{1e-1}{80}$

I really appreciate any help.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.