Issues with training SSD on own dataset

Question

I'm new to ML and trying to train a SSD300, with some Keras-Code github.com/pierluigiferrari/ssd_keras I found on github. For training I'm using an own (very small) dataset of objects that are not in any of the bigger known datasets. My dataset has the following characteristics:

• objects have very different sizes in images (from around 20x40 to 250x200)
• there is only one class labeld in the images
• images are in RGB
• all images are sized to fit in 300x300
• dataset contains 319 images for training and validation

Now my problem is, that the loss-function for validation doesn't converge, but training loss does. See this image showing the loss functions over the epochs. I trained 120 epochs with 1000 steps each: When I try to use the trained weights, coming out of this training, I get zero detections in image. It seems like the model didn't learn anything. I'm using pretrained weights for the underlaying VGG-16 network provided in the github-repository. It is trained on imagenet dataset. My parameters are as follows:

img_height = 300 # Height of the model input images
img_width = 300 # Width of the model input images
img_channels = 3 # Number of color channels of the model input images
mean_color = [123, 117, 104] # The per-channel mean of the images in the dataset. Do not change this value if you're using any of the pre-trained weights.
swap_channels = [2, 1, 0] # The color channel order in the original SSD is BGR, so we'll have the model reverse the color channel order of the input images.
n_classes = 1 # Number of positive classes, e.g. 20 for Pascal VOC, 80 for MS COCO
scales_pascal = [0.1, 0.2, 0.37, 0.54, 0.71, 0.88, 1.05] # The anchor box scaling factors used in the original SSD300 for the Pascal VOC datasets
scales_coco = [0.07, 0.15, 0.33, 0.51, 0.69, 0.87, 1.05] # The anchor box scaling factors used in the original SSD300 for the MS COCO datasets
scales = scales_pascal
aspect_ratios = [[1.0, 2.0, 0.5],
                 [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                 [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                 [1.0, 2.0, 0.5, 3.0, 1.0/3.0],
                 [1.0, 2.0, 0.5],
                 [1.0, 2.0, 0.5]] # The anchor box aspect ratios used in the original SSD300; the order matters
two_boxes_for_ar1 = True
steps = [8, 16, 32, 64, 100, 300] # The space between two adjacent anchor box center points for each predictor layer.
offsets = [0.5, 0.5, 0.5, 0.5, 0.5, 0.5] # The offsets of the first anchor box center points from the top and left borders of the image as a fraction of the step size for each predictor layer.
clip_boxes = False # Whether or not to clip the anchor boxes to lie entirely within the image boundaries
variances = [0.1, 0.1, 0.2, 0.2] # The variances by which the encoded target coordinates are divided as in the original implementation
normalize_coords = True

Can someone please help me by pointing out these questions:

1. How to interpret the loss function? Is it because of the small dataset or because of wrong parameters or something else?
2. Do I have to train my own classifier (VGG-16) or can I use the pretrained one even when my objects don't appear in the pretrained dataset?
3. Do I have to train for a longer time? Means for more epochs?

As additional information: I already trained a faster R-CNN model with the exact same dataset. It worked quiet good and gives me good results.

I would appreciate any help you can provide!

Answer 1

What you are experiencing is called overfitting and it happens because of your very small dataset.

All the model cares about is performance on the training dataset, so given the opportunity, it will simply attempt to memorize it. This is what happens in you case, you feed a model which contains over 130 Million parameters less than 319 images.

So regarding your questions:

1. The loss function shows a clear case of overfitting.
2. On general, it is okay to use a trained model, especially when you only have a small dataset, but in your case, the dataset is too small for any deep-learning model. When I say small dataset, I mean 10k images, not several hundreds.
3. You should not train for longer time, once the validation loss stops improving, it is a clear sign to stop. There is even a training technique named "early stopping" which is designed to stop training once the validation loss stops to drop.

You have to understand that currently, your dataset of 300 images, is irrelevant to the world of deep-learning. So if you still want to use it for object detection, you need to revert to more classic computer-vision techniques like using HOG or SIFT features, or even manually engineering the features for your special case.