# Data Augmentation recommended pipeline

I want to perform image classification using Keras and a dataset made of 50 classes. At the moment, I have only 7 images per class and I need to perform data augmentation in order to train the model and obtain acceptable accuracy values.

I am using the ImageDataGenerator class from keras which is recommended for image augmentation on the fly (during training). Since the classification is performing badly, I was wondering if it would be necessary to perform offline augmentation, i.e, enlarge the dataset before the training, because I honestly think that 7 is far from being a reasonable number of images per class.

Is it a common practice to perform both types of augmentation (before and during the training)? I am planning to use some 3rd party software or tools like imgaug to enlarge the dataset first and save the augmented images to disk and only then perform real-time augmentation with ImageDataGenerator class.

In conclusion, the flow would be similar to this:

• Image pre-processing and offline data augmentation => enlarge the original dataset
• Training with real-time augmentation => Load the dataset and use ImageDataGenerator

What do you think?

Thank you.

• I would recommend adding a description of the images (or an example). Your question is quite specific, but chances are that generic augmentation is not helping enough (for your small dataset). May 24 '19 at 8:51

## 2 Answers

Out of the two pipelines you mentioned, I'd recommend the second (i.e. real-time augmentation). This is better than the first, because by performing random augmentations the network sees different images at each epoch.

I'd recommend imgaug, which is a python library for performing data augmentation. I've found it very helpful as it can work with keras' ImageDataGenerator very well. The way can do this is:

from imgaug import augmenters as iaa

seq = iaa.Sequential([...])  # list of desired augmentors

ig = ImageDataGenerator(preprocessing_function=seq.augment_image)  # pass this as the preprocessing function

gen = ig.flow_from_directory(data_dir)  # nothing else changes with the generator


One final note I'd like to make is that $7 \cdot 50 = 350$ images are very few for deep learning, even with augmentation. Make sure you use a pre-trained network or else you will have a serious overfitting problem.

• What you are saying in the last paragraph is exactly the reason why I posted. The fact that I have a very small amount of available data made me consider about having both pipelines. After the first pipeline, I would kind of create my new dataset, which would then be considered for the second one. Besides the need of more data before feeding the model, I also need to crop some of the images using OpenCV, so this pre-processing step would contain both the enlargement of the dataset and the cropping of the images when needed! You were very helpful, thank you so much! Sep 1 '18 at 1:53
• You can include as many steps of augmentation as you like, but you should always, in my opinion, do them in real-time. The sequential augmentor allows you to select many steps (e.g. first left-right flip, then random translation, then brightness adjustment) with a given probability for each. You should only do the first pipeline as a preprocessing step not an augmentation one (e.g. you have a facial recognition task and you first want to keep just the heads from the images). The difference is that when you do this you should throw away the original dataset in favor of the cropped one. Sep 1 '18 at 12:12
• Done! I can't give you an upvote, sorry! :D Sep 3 '18 at 23:07

Elastic Transformation worked really well on my defect detector: https://github.com/nyck33/defect_detector_CNN_Keras/blob/master/ElasticTransform.ipynb

It's on Kaggle too: https://www.kaggle.com/bguberfain/elastic-transform-for-data-augmentation

It's used for medical imagery because it retains the position of pixels relative to an underlying grid while distorting the image but it works for pretty much any dataset I think.