# trying to decrease overfitting with regularisation in CNN

I am doing transfer learning by retraining the publicly available inception layer, without regularisation here are my initial parameters and results:

training steps: 20,000
learning rate: 0.075
test accuracy :72.9%
train accuracy for final iteration : 98.0%
val accuracy for final iteration : 75.0%


clearly overfitting, so I tried L2 regularization. Here are my parameters and results, for the highest accuracy until now :

training steps: 40,000
learning rate: 0.1
test accuracy :71.1%
alpha for L2 : 0.00075
train accuracy for final iteration : 91.0%
val accuracy for final iteration : 71.0%


Looked promising so I decided to go further with the following hoping to get better accuracy :

training steps: 80,000
learning rate: 0.075
alpha for L2 : 0.005


go the following graph :

Since, the graph was getting saturated so I quit at 50,000 steps.

My question :

Should I continue with it, hoping it will get better or should I try with other values(Please suggest some good range of parameters that I should try)? or may be other techniques like L1 regularization or dropout ?

Note : Since, it is real-world data so getting new data would be difficult, and I cant flip the image to generate new data since orientation is also a factor to determine the label for output classes

• Use drop out, it better performance than L regularizations. May 6, 2018 at 11:01
• What batch size are you using? If it is small (e.g. less than 20), this would cause the large variance errors over iterations. Aug 4, 2018 at 15:31

Steps for overcoming overfitting are the followings:

 1. Add more data
2. Use data augmentation
3. Use architectures that generalize well