Loss decreases, but Validation Loss just fluctuates

Question

I've been trying to implement object detection using a CNN architecture like this:

model = keras.Sequential([
    keras.layers.Input(shape=(320, 320, 1)),
    keras.layers.Conv2D(filters=16, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=2),
    keras.layers.Conv2D(filters=32, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=2),
    keras.layers.Conv2D(filters=64, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=2),
    keras.layers.Conv2D(filters=128, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=2),
    keras.layers.Conv2D(filters=256, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=2),
    keras.layers.Conv2D(filters=512, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.MaxPool2D((2, 2), strides=1, padding="same"),
    keras.layers.Conv2D(filters=1024, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.Conv2D(filters=1024, kernel_size=(3, 3), activation="leaky_relu", padding="same"),
    keras.layers.Conv2D(filters=5, kernel_size=(1, 1), activation="relu", padding="same"),
]);

model.compile(optimizer=keras.optimizers.Adam(learning_rate=0.000005), loss=yolo_loss, run_eagerly=True);

However, while the loss seems to decrease nicely, the validation loss only fluctuates around 300. Loss vs Val Loss

This model is trained on a dataset of 250 images, where 200 are actually used for training while 50 are used for cross-validation. Why could this be? Could my model be too deep? Do I need to reduce my learning rate even more? Or do I just not have enough data?

For reference I am trying to mimic the Tiny YoloV2 architecture shown here

Answer 1

It looks like your model is overfitting: it's learning from the training dataset, but this learning doesn't apply to the test dataset. You can try to reduce the complexity of the model by simplifying your model -fewer layers, fewer neurons, fewer filters, etc- or by adding regularization -l1, l2, dropout, etc.

Answer 2

You have 250 images as training set, and you are using a model with millions of parameters... I'm pretty sure that your model is just memorizing the training set, aka you are overfitting.

At this point you can use many regularization, from prior on weights, drop out, simplifying your model, adding noise to the input, creating new data by applying transformation to your available data (rotation, translation, scale)