I have started a project of classifying a dataset using deep learning. I have tried transfer learning on pretrained models. Now, I want to design a CNN model which can do this work of classification but everywhere it says it needs large data. My question is: Is it possible to design a deep CNN model on an image dataset of size less than 1000 images?
According to your description of the data, it is highly probable that training any neural network of reasonable size will overfit the training data.
One option is to apply any possible regularization approach; these come to mind:
- Use L1/L2 weight regularization.
- Reduce the number of parameters as much as you can (less layers, less number of filters).
- Use dropout.
Also, try any data augmentation trick: rotate images, mirror images, crop and scale images, etc.
As ncasas answered, layer stacking and lesser number of filters do help. I didn't use dropout as I have very few parameters. Regularization is my next approach. I used Batch Normalization to generalize the model. It works very well.
Without Batch Normalization, model overfitted around 65% but using a single Batch Normalization generalized well upto 70%. Use it after convolutional layer for better results.