# How to improve loss and avoid overfitting

I'm trying to build a 2 class image classifier using the architecture suggested in first part of this blog https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html

My dataset has 1500 images of class1 and 500 images of class2. I created 4 copies of class2 to make the number of images same in both classes.

I also use ImageDataGenerator to augment images

datagen = ImageDataGenerator(
featurewise_center=True,
featurewise_std_normalization=True,
shear_range=0.2,
zoom_range=0.5,
rotation_range=45,
horizontal_flip=True,
vertical_flip=True)
datagen.fit(train_imgs)


For first few epochs(about 20) training and validation errors keep reducing until log loss reaches about 0.4 (best I have got till now) after that the model starts to overfit and validation loss keeps increasing.

I know I can prevent overfitting by reducing the network complexity and adding dropouts but that reduces the training accuracy too.

Please suggest some tips to improve the accuracy and avoid overfitting.

• Have you tried tuning L1 and L2 regularization? – Gaius Apr 9 '18 at 20:07
• Didn't try that. Should I try L1 L2 with or with out dropout and and batchnormalization ? – Amit Khanna Apr 9 '18 at 20:32

I am not familiar with the software you are using but keep in mind: You EXPECT accuracy to drop if you reduce over fitting. It is not a bad thing. Over-fitting is essentially "fake accuracy".

Some good approaches in general to avoid over-fitting though: Use cross-validation, normalize your features, increase size of data-set and dont just increase your data-set by copying data.

• Just to be clear, you expect training accuracy to decrease when you reduce overfitting. Test accuracy will increase, which is, of course, the whole point. – Ray Apr 11 '18 at 22:25
• Isn't cross-validation for assessing a model? How it could imporove the performance of the model? – Sherzod May 8 at 7:53

There are a few things you can do to reduce over-fitting.

1. Use Dropout increase its value and increase the number of training epochs
2. Increase Dataset by using Data augmentation
3. Tweak your CNN model by adding more training parameters. Reduce Fully Connected Layers.
4. Change the whole Model
5. Use Transfer Learning (Pre-Trained Models)
• Can you explain what adding more training parameters means? – Anshuman Kumar Apr 23 at 7:52

You can use dropout which will help in controlling the model to over train. While using CNN,data-set plays an important role.The more data-set the more model can learn features from it. You can divide your data-set in 3 parts. Training ,testing and validation.

Here are few things you can try to reduce overfitting:

1. Use batch normalization
8. USE callbacks tf.keras.callbacks.ReduceLROnPlateau here