Hello, during training of one of my models, I observe the following training (blue) and test (orange) loss patterns. At first, the training loss increases, then bends and starts decreasing. Just wondering what might be a very general cause of this, as it seems to be rather unusual? Am I using the wrong type of initialization? I am using convolutional layers, initialized with Xavier uniform in pytorch. After the initial bending, the train and validation loss nicely decrease.
Thanks!
