I am doing transfer learning by retraining the publicly available inception layer, without regularisation here are my initial parameters and results:
training steps: 20,000 learning rate: 0.075 test accuracy :72.9% train accuracy for final iteration : 98.0% val accuracy for final iteration : 75.0%
clearly overfitting, so I tried L2 regularization. Here are my parameters and results, for the highest accuracy until now :
training steps: 40,000 learning rate: 0.1 test accuracy :71.1% alpha for L2 : 0.00075 train accuracy for final iteration : 91.0% val accuracy for final iteration : 71.0%
Looked promising so I decided to go further with the following hoping to get better accuracy :
training steps: 80,000 learning rate: 0.075 alpha for L2 : 0.005
go the following graph :
Since, the graph was getting saturated so I quit at 50,000 steps.
My question :
Should I continue with it, hoping it will get better or should I try with other values(Please suggest some good range of parameters that I should try)? or may be other techniques like L1 regularization or dropout ?
Note : Since, it is real-world data so getting new data would be difficult, and I cant flip the image to generate new data since orientation is also a factor to determine the label for output classes