0
$\begingroup$

I have started a project of classifying a dataset using deep learning. I have tried transfer learning on pretrained models. Now, I want to design a CNN model which can do this work of classification but everywhere it says it needs large data. My question is: Is it possible to design a deep CNN model on an image dataset of size less than 1000 images?

$\endgroup$
11
  • $\begingroup$ Transfer learning (pre-training on a large generic dataset) is the typical approach for these cases. Isn't transfer learning an acceptable approach in this case? $\endgroup$ – noe Mar 13 '18 at 13:12
  • $\begingroup$ It is acceptable but we have to do something new to compare and show that it is competing with those results. So making a new model is the only possible way for me. $\endgroup$ – Saumya Sambit Acharya Mar 13 '18 at 13:15
  • $\begingroup$ Then why not just training the same model from scratch on those 1000 images? $\endgroup$ – noe Mar 13 '18 at 13:33
  • $\begingroup$ For example let's take GoogleNet. So many parameters but images are total 1000. Won't learning from scratch make it definitely overfit? $\endgroup$ – Saumya Sambit Acharya Mar 13 '18 at 13:37
  • $\begingroup$ Of course, but isn't the purpose of the new model just gauge the performance of the model trained using transfer learning vs. not using it? Or do you just want to try other approaches that might be effective with a small data set? $\endgroup$ – noe Mar 13 '18 at 13:45
1
$\begingroup$

According to your description of the data, it is highly probable that training any neural network of reasonable size will overfit the training data.

One option is to apply any possible regularization approach; these come to mind:

  • Use L1/L2 weight regularization.
  • Reduce the number of parameters as much as you can (less layers, less number of filters).
  • Use dropout.

Also, try any data augmentation trick: rotate images, mirror images, crop and scale images, etc.

$\endgroup$
0
1
$\begingroup$

As ncasas answered, layer stacking and lesser number of filters do help. I didn't use dropout as I have very few parameters. Regularization is my next approach. I used Batch Normalization to generalize the model. It works very well.

Without Batch Normalization, model overfitted around 65% but using a single Batch Normalization generalized well upto 70%. Use it after convolutional layer for better results.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.