I am using the following augmentations on dataset of size 9 GB:
datagen = ImageDataGenerator(
featurewise_center=False, # set input mean to 0 over the dataset
samplewise_center=False, # set each sample mean to 0
featurewise_std_normalization=False, # divide inputs by std of the dataset
samplewise_std_normalization=False, # divide each input by its std
zca_whitening=True, # apply ZCA whitening
rotation_range=30, # randomly rotate images in the range (degrees, 0 to 180)
width_shift_range=0.1, # randomly shift images horizontally (fraction of total width)
height_shift_range=0.1, # randomly shift images vertically (fraction of total height)
horizontal_flip=True, # randomly flip images
vertical_flip=True) # randomly flip images
datagen.fit(a)
model.fit_generator(datagen.flow(a,b, batch_size=32),
steps_per_epoch=len(a) / 32, epochs=epochs, class_weight = sclass_weight, validation_data = [c, d],callbacks = [MetricsCheckpoint('logs')])
When the code comes to the datagen.fit
, I get into memory error ( the code doesn't even go into training)
I have 50 gb ram and am training it on a K80 with a batch size of 32, so don't think that will be a problem.
It works fine when I comment all the augmentations.
Can someone tell me where I am going wrong?