1
$\begingroup$

I'm working on a multi-classification deep learning algorithm and I was getting big over-fitting: enter image description here

My model is supposed to classify sunglasses on 17 different brands, but I only had around 400 images from each brand so I created a folder with data augmented x3 times, generating images with these parameters:

datagen = ImageDataGenerator(
    rotation_range=30,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    fill_mode='nearest')

After doing so i got these results:

enter image description here

I don't know if it's correct to do the validation only using the original images or if I have to use also the augmented images for the validation, also is strange for me to get higher accuracy on the validation than the training.

$\endgroup$
1
$\begingroup$

You should validate only on the original images. The augmentation is there so that it can help your model generalize better, but to evaluate your model you need actual images, not transformed ones.

To do this in keras you need to define two instances of the ImageDataGenerator, one for training and one for validating. To train the model you need to set both generators to the fit_generator function.

train_gen = ImageDataGenerator(aug_params).flow_from_directory(train_dir)
valid_gen = ImageDataGenerator().flow_from_directory(valid_dir)

model.fit_generator(train_gen, validation_data=valid_gen)

It is possible to achieve a higher validation accuracy than a train accuracy if you heavily augment the training data.

$\endgroup$
  • 1
    $\begingroup$ Thanks so much for your response, I actually was using something similar but with data augmentation in the validation, this clarify what i'm supposed to do. $\endgroup$ – Santiago Marin Mejia Nov 21 '18 at 14:53
  • 1
    $\begingroup$ Also I think you have to add this to the code to get the validation part: valid_gen = ImageDataGenerator().flow_from_directory(train_dir, validation_split=0.2) This validation_split=0.2 if not used the model will use it validation_split=0 as default $\endgroup$ – Santiago Marin Mejia Nov 21 '18 at 15:16
  • $\begingroup$ Yeah I meant to write a different directory containing the validation set. Thanks for pointing it out. $\endgroup$ – Djib2011 Nov 21 '18 at 18:32
2
$\begingroup$

Ideally, data augmentation is a step in your training pipeline, which comes after splitting your data into train/validation/test sets. Otherwise, you have the same data point in both training and testing, even if it a little rotated.

So your training pipeline could be something like this:

          +-> training set ---> data augmentation --+
          |                                         |
          |                                         +-> model training --+
          |                                         |                    |
all data -+-> validation set -----------------------+                    |
          |                                                              +-> model testing
          |                                                              |
          |                                                              |
          +-> test set --------------------------------------------------+
$\endgroup$
  • $\begingroup$ thanks so much, with your permission I will use this graphic representation of my training pipeline in my presentation. $\endgroup$ – Santiago Marin Mejia Nov 21 '18 at 14:57
  • $\begingroup$ @SantiagoMarinMejia glad I could be of help. Feel free to use it ;) $\endgroup$ – BrunoGL Nov 22 '18 at 15:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.