I'm having issues with my CNN, using Keras with Theano backend. Basically, I need to classify 340x340 grayscale images into 6 categories. The problem is my CNN gives too "hard" probabilities, for instance it will rarely give predictions with some uncertainty, and always tries to push for a 90%+ for one class. The problem is that for my coursework, the penalty used is very harsh for complete miss classification, and uncertainty is much preferred. ( so having a prediction like [0.6, 0.3, 0.2, ...] is much better than having [0.9,0.03,0.02,..].
I'm unsure why this is happening. My dataset consists of 2400 images, which are from different CCTV, and task is about recognising possible objects. Only 800 of the samples are actually from the data, the other 1600 have been generated through data augmentation. Note that it is therefore extremely likely a that some pictures are either identical, or extremely similar (e.g. the same scene, one second later)
model = Sequential()
#1
# Few filter to take big stuff out
# Also, first layer is not conv so that I can reuse that layer separately
model.add(Dropout(0.1, input_shape=(1,340,340)))
model.add(Convolution2D(64, 4, 4, border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2), dim_ordering="th"))
#2
model.add(Dropout(0.1))
model.add(Convolution2D(128, 4, 4, border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2), dim_ordering="th"))
#3
model.add(Dropout(0.1))
model.add(Convolution2D(256, 4, 4, border_mode='same'))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2), dim_ordering="th"))
#4
model.add(Flatten())
model.add(Dense(1024))
model.add(Activation('relu'))
#5
model.add(Dense(512))
model.add(Activation('relu'))
#6
model.add(Dense(6))
model.add(Activation('softmax'))
opt = SGD(lr=0.001)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
print "Training.."
filepath = "log/weights-improvement-{epoch:02d}---{val_acc:.2f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='val_acc', verbose=1, save_best_only=True, mode='max')
callbacks_list = [checkpoint]
model.fit(X_t, y_t, validation_split=0.1, nb_epoch=500, batch_size=32, callbacks=callbacks_list)
How do you suggest I fix this? Thank you in advance!