I'm new to deep learning and I recently trained a simple convolutional neural network from Francois Chollet's Deep Learning with Python book. The network was trained on 12500 images of cats and dogs. I used 6250 images in the validation set.
For my own education, I trained the network overnight for 1200 epochs and found that the training loss, validation loss and validation accuracy eventually stopped changing. See the charts below.
My question is the following: Why does the training loss, validation loss and validation accuracy eventually stop changing?
Note that in the "Training and Validation Loss" chart above, both training and validation loss dips down to a minimum near the start of training before rising, then plateauing at about 200 epochs. This indicates that the cost function does not remain "stuck" at its minimum. Perhaps the cost function gets stuck at some local minima because the cost function is non-convex? Maybe I can resolve the issue by adjusting the learning rate manually?
I should also mention that I find it strange that the training loss becomes constant after about 200 epochs, even though I used data augmentation and the training accuracy never goes to a constant value.
The following lists some additional details about the model setup:
Batch Size: 64
from tensorflow import keras from tensorflow.keras import layers data_augmentation = keras.Sequential( [ layers.experimental.preprocessing.RandomFlip("horizontal"), layers.experimental.preprocessing.RandomRotation(0.1), layers.experimental.preprocessing.RandomZoom(0.2), ] )
model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
Please feel free to ask for more details.