Training Accuracy stuck in Keras

Question

I have trained a CNN using keras for Image classification with 3 classes. The results are bad and I'm trying to understand what the classifier has learnt and what it has not. It's only giving me an output of 1 class.

I have made changes to the learning rate, activation(relu, sigmoid and softmax for last layer), changed the architecture and the optimizer(SGD and Adam) but the training accuracy is stuck at ~33.33%. It's definitely not a coincidence because I only have 3 classes.

My present architecture is

[Conv -> Relu -Conv -> Relu -> MAxPool] * 3 -> Flatten -> [Dense -> Relu] * 2 -> Dense -> Softmax

My first 2 conv layers have 64 filters of size (3, 3) and the remaining conv layers have 32 filters of the same size.

My Dense layer goes like this 128 units-> 64 units -> 3 units.

I started with a simple model and made it more complex to improve it. But there has been no improvement after any of these changes.

I have used two activations 'relu' and 'sigmoid' for experimental purposes. I'm thinking of only using sigmoid and softmax for the last layer.

I have ~13000 images to train and 1400 for validation. The distribution is almost equal among the 3 classes.

I was using this syntax to add the activation. The summary didn't show any activation layers. And my network wasn't improving.

classifier = Sequential()
    classifier.add(Conv2D(32, (3, 3), input_shape  = (256, 256, 3), activation = 'relu'))
    classifier.add(Conv2D(32, (3, 3), activation = 'relu'))
    classifier.add(MaxPooling2D(pool_size = (2,2)))

    _________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_11 (Conv2D)           (None, 254, 254, 32)      896       
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 252, 252, 32)      9248      
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 84, 84, 32)        0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 82, 82, 32)        9248      
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 80, 80, 32)        9248      
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 40, 40, 32)        0         
_________________________________________________________________
conv2d_15 (Conv2D)           (None, 38, 38, 32)        9248      
_________________________________________________________________
conv2d_16 (Conv2D)           (None, 36, 36, 32)        9248      
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 18, 18, 32)        0         
_________________________________________________________________
flatten_1 (Flatten)          (None, 10368)             0         
_________________________________________________________________
dense_6 (Dense)              (None, 128)               1327232   
_________________________________________________________________
dense_7 (Dense)              (None, 64)                8256      
_________________________________________________________________
dense_8 (Dense)              (None, 3)                 195       
=================================================================
Total params: 1,382,819
Trainable params: 1,382,819
Non-trainable params: 0

Edit: updated Network. But when I add Activation as a new Layer the architecture changes. And now my network seems to work.

classifier = Sequential()
classifier.add(Conv2D(64, (3, 3), input_shape  = (256, 256, 3))
classifier.add(Activation('relu'))
classifier.add(Conv2D(64, (3, 3))
classifier.add(Activation('relu'))
classifier.add(MaxPooling2D(pool_size = (2,2)))

---

Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_13 (Conv2D)           (None, 254, 254, 32)      896       
_________________________________________________________________
activation_1 (Activation)    (None, 254, 254, 32)      0         
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 252, 252, 32)      9248      
_________________________________________________________________
activation_2 (Activation)    (None, 252, 252, 32)      0         
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 84, 84, 32)        0         
_________________________________________________________________
conv2d_15 (Conv2D)           (None, 82, 82, 32)        9248      
_________________________________________________________________
activation_3 (Activation)    (None, 82, 82, 32)        0         
_________________________________________________________________
conv2d_16 (Conv2D)           (None, 80, 80, 32)        9248      
_________________________________________________________________
activation_4 (Activation)    (None, 80, 80, 32)        0         
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 40, 40, 32)        0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 51200)             0         
_________________________________________________________________
dense_7 (Dense)              (None, 128)               6553728   
_________________________________________________________________
activation_5 (Activation)    (None, 128)               0         
_________________________________________________________________
dense_8 (Dense)              (None, 3)                 387       
_________________________________________________________________
activation_6 (Activation)    (None, 3)                 0         
=================================================================
Total params: 6,582,755
Trainable params: 6,582,755
Non-trainable params: 0

Directory structure:

Training_path
    -Label1 Folder
    -Label2 Folder
    -Label3 Folder

I think that was the problem in my network. That the argument activation wasn't working as expected and no activations were performed on the network input. What I don't understand is that both syntax's are equivalent(according to the documentation) and yet are producing different results.

Answer 1

So there were two problems:

1. When I tried to add the activations as a parameter Conv2d( activation = 'relu'), my activation layers were not added as displayed in the summary in my question. But by usingadd(Activation('relu'))`, it worked as shown in the question.
2. But that didn't solve the problem. I used learning rates of 0.01, 0.007, 0.005 which showed good training and validation accuracies for few epochs but plummeted to 33.33 again so probably overshooting. So I used a much lower learning rates of 0.0001, which worked pretty well.

Answer 2

I faced a similar issue. One-hot encoding the target variable using nputils in Keras, solved the issue of accuracy and validation loss being stuck. Using weights for balancing the target classes further improved performance.

y_train_temp = y_train.copy()
y_val_temp = y_val.copy()
from keras.utils.np.utils import to_categorical
y_train = to_categorical(y_train)
y_val = to_categorical(y_val)