I noticed something when looking at the output of the verbose. When I train my model, in the early part of the epoch (first 20 %), the loss is decreasing a lot. And then in the rest of the epoch (last 80%), the loss is very stable and doesn't change that much until the next epoch. It does the same thing.
I build a model that is training a kind of large dataset (60000 entries). I am using Keras and Tensorflow and my model is just a simple regression model with conv2d and dense layers. I am trying to get the best loss function possible. The loss function that I am using is just a simple Mean Squared Error (MSE). I am also using Adam optimizer.
I don't understand how the loss is decreasing a lot at the beginning of the epoch and less for the rest of the training. Should I reduced the size of my database ? Is it considered overtraining then ?
Is there a way to speed up this process (like early stopping but just for the epoch) ? As I see it most of the training is done for the first part of the data. Maybe I am wrong.