I am using the following augmentations on dataset of size 9 GB:
datagen = ImageDataGenerator( featurewise_center=False, # set input mean to 0 over the dataset samplewise_center=False, # set each sample mean to 0 featurewise_std_normalization=False, # divide inputs by std of the dataset samplewise_std_normalization=False, # divide each input by its std zca_whitening=True, # apply ZCA whitening rotation_range=30, # randomly rotate images in the range (degrees, 0 to 180) width_shift_range=0.1, # randomly shift images horizontally (fraction of total width) height_shift_range=0.1, # randomly shift images vertically (fraction of total height) horizontal_flip=True, # randomly flip images vertical_flip=True) # randomly flip images datagen.fit(a) model.fit_generator(datagen.flow(a,b, batch_size=32), steps_per_epoch=len(a) / 32, epochs=epochs, class_weight = sclass_weight, validation_data = [c, d],callbacks = [MetricsCheckpoint('logs')])
When the code comes to the
datagen.fit, I get into memory error ( the code doesn't even go into training)
I have 50 gb ram and am training it on a K80 with a batch size of 32, so don't think that will be a problem.
It works fine when I comment all the augmentations.
Can someone tell me where I am going wrong?