0
$\begingroup$

enter image description here

Hello, during training of one of my models, I observe the following training (blue) and test (orange) loss patterns. At first, the training loss increases, then bends and starts decreasing. Just wondering what might be a very general cause of this, as it seems to be rather unusual? Am I using the wrong type of initialization? I am using convolutional layers, initialized with Xavier uniform in pytorch. After the initial bending, the train and validation loss nicely decrease.

Thanks!

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.