I have trained a CNN using keras for Image classification with 3 classes. The results are bad and I'm trying to understand what the classifier has learnt and what it has not. It's only giving me an output of 1 class.
I have made changes to the learning rate, activation(relu, sigmoid and softmax for last layer), changed the architecture and the optimizer(SGD and Adam) but the training accuracy is stuck at ~33.33%. It's definitely not a coincidence because I only have 3 classes.
My present architecture is
[Conv -> Relu -Conv -> Relu -> MAxPool] * 3 -> Flatten -> [Dense -> Relu] * 2 -> Dense -> Softmax
My first 2 conv layers have 64 filters of size (3, 3) and the remaining conv layers have 32 filters of the same size.
My Dense layer goes like this 128 units-> 64 units -> 3 units.
I started with a simple model and made it more complex to improve it. But there has been no improvement after any of these changes.
I have used two activations 'relu' and 'sigmoid' for experimental purposes. I'm thinking of only using sigmoid and softmax for the last layer.
I have ~13000 images to train and 1400 for validation. The distribution is almost equal among the 3 classes.
I was using this syntax to add the activation. The summary didn't show any activation layers. And my network wasn't improving.
classifier = Sequential()
classifier.add(Conv2D(32, (3, 3), input_shape = (256, 256, 3), activation = 'relu'))
classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2,2)))
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d_11 (Conv2D) (None, 254, 254, 32) 896
_________________________________________________________________
conv2d_12 (Conv2D) (None, 252, 252, 32) 9248
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 84, 84, 32) 0
_________________________________________________________________
conv2d_13 (Conv2D) (None, 82, 82, 32) 9248
_________________________________________________________________
conv2d_14 (Conv2D) (None, 80, 80, 32) 9248
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 40, 40, 32) 0
_________________________________________________________________
conv2d_15 (Conv2D) (None, 38, 38, 32) 9248
_________________________________________________________________
conv2d_16 (Conv2D) (None, 36, 36, 32) 9248
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 18, 18, 32) 0
_________________________________________________________________
flatten_1 (Flatten) (None, 10368) 0
_________________________________________________________________
dense_6 (Dense) (None, 128) 1327232
_________________________________________________________________
dense_7 (Dense) (None, 64) 8256
_________________________________________________________________
dense_8 (Dense) (None, 3) 195
=================================================================
Total params: 1,382,819
Trainable params: 1,382,819
Non-trainable params: 0
Edit: updated Network. But when I add Activation as a new Layer the architecture changes. And now my network seems to work.
classifier = Sequential()
classifier.add(Conv2D(64, (3, 3), input_shape = (256, 256, 3))
classifier.add(Activation('relu'))
classifier.add(Conv2D(64, (3, 3))
classifier.add(Activation('relu'))
classifier.add(MaxPooling2D(pool_size = (2,2)))
Layer (type) Output Shape Param #
=================================================================
conv2d_13 (Conv2D) (None, 254, 254, 32) 896
_________________________________________________________________
activation_1 (Activation) (None, 254, 254, 32) 0
_________________________________________________________________
conv2d_14 (Conv2D) (None, 252, 252, 32) 9248
_________________________________________________________________
activation_2 (Activation) (None, 252, 252, 32) 0
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 84, 84, 32) 0
_________________________________________________________________
conv2d_15 (Conv2D) (None, 82, 82, 32) 9248
_________________________________________________________________
activation_3 (Activation) (None, 82, 82, 32) 0
_________________________________________________________________
conv2d_16 (Conv2D) (None, 80, 80, 32) 9248
_________________________________________________________________
activation_4 (Activation) (None, 80, 80, 32) 0
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 40, 40, 32) 0
_________________________________________________________________
flatten_2 (Flatten) (None, 51200) 0
_________________________________________________________________
dense_7 (Dense) (None, 128) 6553728
_________________________________________________________________
activation_5 (Activation) (None, 128) 0
_________________________________________________________________
dense_8 (Dense) (None, 3) 387
_________________________________________________________________
activation_6 (Activation) (None, 3) 0
=================================================================
Total params: 6,582,755
Trainable params: 6,582,755
Non-trainable params: 0
Directory structure:
Training_path
-Label1 Folder
-Label2 Folder
-Label3 Folder
I think that was the problem in my network. That the argument activation wasn't working as expected and no activations were performed on the network input. What I don't understand is that both syntax's are equivalent(according to the documentation) and yet are producing different results.