I created a multi-scale CNN in python keras. The network architecture is similar to the diagram. Here, same image is fed to 3 CNN's with different architectures. The weights are NOT shared.
I coded the following multiscale CNN in keras which loosely resembles the architecture in the diagram. But I keep getting "Out of memory ERROR" even when the train_dir has 2 images. Would appreciate help...
#main CNN model - CNN1
main_model = Sequential()
main_model.add(Convolution2D(32, 3, 3, input_shape=(3, 224, 224)))
main_model.add(Activation('relu'))
main_model.add(MaxPooling2D(pool_size=(2, 2)))
main_model.add(Convolution2D(32, 3, 3))
main_model.add(Activation('relu'))
main_model.add(MaxPooling2D(pool_size=(2, 2)))
main_model.add(Convolution2D(64, 3, 3))
main_model.add(Activation('relu'))
main_model.add(MaxPooling2D(pool_size=(2, 2))) # the main_model so far outputs 3D feature maps (height, width, features)
main_model.add(Flatten())
#lower features model - CNN2
lower_model1 = Sequential()
lower_model1.add(Convolution2D(32, 3, 3, input_shape=(3, 224, 224)))
lower_model1.add(Activation('relu'))
lower_model1.add(MaxPooling2D(pool_size=(2, 2)))
lower_model1.add(Flatten())
#lower features model - CNN3
lower_model2 = Sequential()
lower_model2.add(Convolution2D(32, 3, 3, input_shape=(3, 224, 224)))
lower_model2.add(Activation('relu'))
lower_model2.add(MaxPooling2D(pool_size=(2, 2)))
lower_model2.add(Flatten())
#merged model
merged_model = Merge([main_model, lower_model1, lower_model2], mode='concat')
final_model = Sequential()
final_model.add(merged_model)
final_model.add(Dense(64))
final_model.add(Activation('relu'))
final_model.add(Dropout(0.5))
final_model.add(Dense(1))
final_model.add(Activation('sigmoid'))
final_model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy'])
print 'About to start training merged CNN'
train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True)
train_generator = train_datagen.flow_from_directory(train_data_dir, target_size=(224, 224), batch_size=32, class_mode='binary')
test_datagen = ImageDataGenerator(rescale=1./255)
test_generator = test_datagen.flow_from_directory(args.test_images, target_size=(224, 224), batch_size=32, class_mode='binary')
final_train_generator = zip(train_generator, train_generator, train_generator)
final_test_generator = zip(test_generator, test_generator, test_generator)
final_model.fit_generator(final_train_generator, samples_per_epoch=nb_train_samples, nb_epoch=nb_epoch, validation_data=final_test_generator, nb_val_samples=nb_validation_samples)
UPDATE 1: Providing more system info
I am using Theano-0.9.0.dev5 | Keras-1.2.1 | Python 2.7.12 | OSX Sierra 10.12.3 (16D32) | Macbook Pro 16GB RAM | CPU mode
~/.keras/keras.json
contents
{
"image_dim_ordering": "th",
"epsilon": 1e-07,
"floatx": "float64",
"backend": "theano"
}
Dont have .theanorc
file
Please note individual CNN models are training fine. Only the merged code above causes issues.
UPDATE 2: on 27th January, 2017. Tried the following -
Reduced the no. of parameters of the CNN from 53 million to 100K. But still no use. The network eventually fails to train due to memory issues.
Reduced the batch size of images training to 8. The network training fails with the same reason.
No working solution at the time of writing this update...
float32
instead offloat64
in the keras configuration? $\endgroup$ – noe Jan 24 '17 at 11:08