6
$\begingroup$

I want to optimize some hyperparameters for a CNN architecture by using GridSearchCV (Scikit-Learn) in combination with Data Augmentation (ImageDataGenerator from Keras). However, GridSearchCV only offers the fit function and not the fit_generator function. Is it even recommended to use data augmentation with GridSearchCV? The parameters for the ImageDataGenerator are already fixed and should not be changed. Would it be better to first determine the hyperparameters via grid search without data augmentation and only to use data augmentation for the final model?

What do you think about this topic? What are your experiences?

Improve this question
$\endgroup$
5
  • $\begingroup$ I don't think so it's possible that way, because a selected set of params will be run through the whole experiment, so you can kinda do it trivially by writing loops over the image_data_generator but I wouldn't recommend it, why not just use the well know working params?[I might be wrong] $\endgroup$
    – Aditya
    Dec 29, 2019 at 18:32
  • $\begingroup$ Sorry, I think I have expressed myself misleadingly. I want to optimize hyperparameters like learning rate, dropout rate etc. with Grid Search. Since the trainings data set is small, I consider whether it would be a good idea to use Data Augmentation already during grid search and not only for training of the final model. $\endgroup$
    – Code Now
    Dec 29, 2019 at 21:33
  • $\begingroup$ I had a typo there, there are packages which can do that for you but it doesn't makes sense to do hyper param when your dataset is so small.., just fine-tune and you should be good enough imho; to do it nevertheless, what I mean was looping over all your possible hyoer-params search space and building models for each possible combination.. $\endgroup$
    – Aditya
    Dec 30, 2019 at 3:39
  • $\begingroup$ I think you can absolutely do it, and is actually a good standard methodology to look for the best possible model (at least applying cross-validation if not grid search for efficiency concerns); I will answer with a worked example doing it from scratch, you do not need to use that 'fit_generator' to do it $\endgroup$
    – German C M
    Dec 31, 2019 at 17:03
  • $\begingroup$ @German C M _That sounds very interesting. I'm looking forward to the example that works without a fit_generator. $\endgroup$
    – Code Now
    Jan 3, 2020 at 10:52

1 Answer 1

Reset to default
1
$\begingroup$

As promised, here you can find an example of how you could apply kfold cross validation for a defined convolutional neural network model, applied to an augmented dataset. You can find the code as a simple gist here It is done as follows:

  1. for a subset of the CIFAR10 images dataset, generate 3 augmented images (by applying horizontal_flip) per original image, so we should finally have as the number of final images in the augmented dataset: 'number of images in the original dataset' * 3.

enter image description here

  1. check that indeed the built augmented dataset has the new expected number of images. We have just created the augmented dataset, not the fit step yet

enter image description here

  1. apply kfold cross validation on the augmented dataset for several hiperparameters combinations; in this example, 3 pairs of 'learning rate-momentum' have been tried. It is made via the usual 'fit' method: enter image description here enter image description here

  2. display the results in a dataframe

enter image description here

This way, we have applied hyperparametrization via kfold cross validation; not a full grid search but only with 3 pairs of hiperparams, but the idea would be the same, not depending on the fit_generator method but making yourself your k folds cross validation on the generated augmented dataset. We could also include other data augmentation strategies in this cross validation.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.