CNN unbalanced and small dataset

Question

I would like to use CNN to make classification with 5 classes, but 4 of these classes only have between 16 and 60 images, while the last one has more than 1300. I know 16 or 60 images are not enough, so I want to use transfer learning, fine tuning and data augmentation. However, I have several questions.

As the data augmentation must only be used for the training data, I will have very few images from the 4 classes for the validation set, would it be a problem?

Is it needed to split it in training/validation/test, or is training/validation enough?

Another problem is the unbalanced data: with such a difference between the number of images in each class, would oversampling or undersampling be a good solution?

For transfer learning and fine tuning, should I freeze all the convolutional layers or should I only train a FC layer?

Answer 1

As the data augmentation must only be used for the training data, I will have very few images from the 4 classes for the validation set, would it be a problem?

You may first perform data augmentation on the image dataset and then split it into train/test or train/test/validation. Just make sure that the images present in the test split are not included in the training split or vice-versa. See this.

Is it needed to split it in training/validation/test, or is training/validation enough ?

The validation dataset is particularly to fine-tune the model's hyperparameters i.e for hyperparameter optimization. If you want to simply skip that procedure, just make two splits, train, and test. See this.

See this answer to know the difference between test and validation dataset.

Another problem is the unbalanced data with such a difference between the number of images in each class, would oversampling or undersampling be a good solution ?

I would encourage you to augment the images for a specific class until you are left with a near-balanced dataset.

For transfer learning and fine tuning, should I freeze all the convolutional layers or should I only train a FC layer?

As a common practice, you may try unfreezing some initial layers ( layers that usually have lesser filters ). Refer to this blog. Deeper layers have more filters and extract high-level features.