0
$\begingroup$

Background

Hello,

I'm new to deep learning and I recently trained a simple convolutional neural network from Francois Chollet's Deep Learning with Python book. The network was trained on 12500 images of cats and dogs. I used 6250 images in the validation set.

For my own education, I trained the network overnight for 1200 epochs and found that the training loss, validation loss and validation accuracy eventually stopped changing. See the charts below.

Main Question

My question is the following: Why does the training loss, validation loss and validation accuracy eventually stop changing?

enter image description here

enter image description here

Some Observations

Note that in the "Training and Validation Loss" chart above, both training and validation loss dips down to a minimum near the start of training before rising, then plateauing at about 200 epochs. This indicates that the cost function does not remain "stuck" at its minimum. Perhaps the cost function gets stuck at some local minima because the cost function is non-convex? Maybe I can resolve the issue by adjusting the learning rate manually?

I should also mention that I find it strange that the training loss becomes constant after about 200 epochs, even though I used data augmentation and the training accuracy never goes to a constant value.

Additional Details

The following lists some additional details about the model setup:

Batch Size: 64

Data Augmentation:

from tensorflow import keras
from tensorflow.keras import layers

data_augmentation = keras.Sequential(
    [
        layers.experimental.preprocessing.RandomFlip("horizontal"),
        layers.experimental.preprocessing.RandomRotation(0.1),
        layers.experimental.preprocessing.RandomZoom(0.2),
    ]
)

Architecture:

enter image description here

Model Compilation:

model.compile(loss='binary_crossentropy',
          optimizer='rmsprop',
          metrics=['accuracy'])

Please feel free to ask for more details.

Regards,

John

$\endgroup$
1
  • 1
    $\begingroup$ it is natural after some point the model has learned all that can learn (with this training, architecture and so on..). Some residual error may exist and that is good as we do not want the model to overfit $\endgroup$ – Nikos M. May 4 at 15:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.