I am training a CNN (SSD Inception V2) and I get a strange shape of the training loss:
At first, I thought it was a too large learning rate (suggested in Question Oscillating loss in CNN ). But after taking a closer look, I don't think that's the case. When I zoom in to where the increase in loss happens, it looks like this:
My interpretation is that this cannot be caused by learning rate because the individual steps made in the training are relatively small. But there is something the causes the training loss to start increasing at some point.
Question: What might be the cause of this? I am not looking for a technical answer or a fix but rather for an answer that would help me understand the behaviour on a conceptual level.