I'm working on a multi-classification deep learning algorithm and I was getting big over-fitting: enter image description here

My model is supposed to classify sunglasses on 17 different brands, but I only had around 400 images from each brand so I created a folder with data augmented x3 times, generating images with these parameters:

datagen = ImageDataGenerator(

After doing so i got these results:

enter image description here

I don't know if it's correct to do the validation only using the original images or if I have to use also the augmented images for the validation, also is strange for me to get higher accuracy on the validation than the training.


4 Answers 4


You should validate only on the original images. The augmentation is there so that it can help your model generalize better, but to evaluate your model you need actual images, not transformed ones.

To do this in keras you need to define two instances of the ImageDataGenerator, one for training and one for validating. To train the model you need to set both generators to the fit_generator function.

train_gen = ImageDataGenerator(aug_params).flow_from_directory(train_dir)
valid_gen = ImageDataGenerator().flow_from_directory(valid_dir)

model.fit_generator(train_gen, validation_data=valid_gen)

It is possible to achieve a higher validation accuracy than a train accuracy if you heavily augment the training data.

  • 1
    $\begingroup$ Thanks so much for your response, I actually was using something similar but with data augmentation in the validation, this clarify what i'm supposed to do. $\endgroup$ Nov 21, 2018 at 14:53
  • 2
    $\begingroup$ Also I think you have to add this to the code to get the validation part: valid_gen = ImageDataGenerator().flow_from_directory(train_dir, validation_split=0.2) This validation_split=0.2 if not used the model will use it validation_split=0 as default $\endgroup$ Nov 21, 2018 at 15:16
  • $\begingroup$ Yeah I meant to write a different directory containing the validation set. Thanks for pointing it out. $\endgroup$
    – Djib2011
    Nov 21, 2018 at 18:32

Ideally, data augmentation is a step in your training pipeline, which comes after splitting your data into train/validation/test sets. Otherwise, you have the same data point in both training and testing, even if it a little rotated.

So your training pipeline could be something like this:

          +-> training set ---> data augmentation --+
          |                                         |
          |                                         +-> model training --+
          |                                         |                    |
all data -+-> validation set -----------------------+                    |
          |                                                              +-> model testing
          |                                                              |
          |                                                              |
          +-> test set --------------------------------------------------+
  • $\begingroup$ thanks so much, with your permission I will use this graphic representation of my training pipeline in my presentation. $\endgroup$ Nov 21, 2018 at 14:57
  • $\begingroup$ @SantiagoMarinMejia glad I could be of help. Feel free to use it ;) $\endgroup$ Nov 22, 2018 at 15:03
  • $\begingroup$ I still don't track why it would make a difference whether you're applying linear transformations to the validation set. As far as I understand, and this may be wrong, the validation set does not feed directly into backprop, and backprop is the only means of biasing the resulting model. So theoretically, any image could be used as validation. Where is the failure in my logic? $\endgroup$
    – iiian
    Jul 20, 2022 at 5:08
  • $\begingroup$ @iiian what are you using your validation set for? $\endgroup$ Jul 20, 2022 at 9:30

You don't need to validate using data augmentation. You are using data aug only for training (because you don't have enough data). If you had much data then there was no point in data augmentation.

And you need data aug to reduce overfitting, there are other methods of reducing overfitting like dropout.


I just want to point out that, though generally, we do not apply data augmentation to the validation data, there is a technique called "test time augmentation". In its simplest form, we make several predictions for each image, some of which are augmented (the same way as training) before making predictions. Then, we ensemble these predictions to make the final prediction.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.