enter image description here

Hello, during training of one of my models, I observe the following training (blue) and test (orange) loss patterns. At first, the training loss increases, then bends and starts decreasing. Just wondering what might be a very general cause of this, as it seems to be rather unusual? Am I using the wrong type of initialization? I am using convolutional layers, initialized with Xavier uniform in pytorch. After the initial bending, the train and validation loss nicely decrease.




Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.