CNN not learning properly

Question

Before marking my question as duplicate, I would like to say that I have tried all the possible solutions mentioned in similar questions, but that doesn't seem to work.

I am currently working on blood cells classification problem where we basically have to classify blood images (4 classes). The dataset consists of 9957 images, nearly equal number of images of all the 4 classes. The accuracy always hovers around 25-27% even after trying different optimizers and learning rates. I even tried training upto 100 epochs. Image augmentation doesn't help. Also, it is not that it is predicting same class for all images although for 1 particular batch of images, it predicts the same class. It again predicts some other class for all images in the next batch. So, I would just like to know, what am I possibly doing wrong? Is the dataset not sufficient, or the architecture should have more hidden layers, or am I not implementing optimizer or loss function correctly or is there any silly mistake I am overlooking in my code ?

My CNN architecture: (fs means filter_size, nf means number of filters, s is no. strides)

Input(80,80,1)->Conv(fs = 3, nf = 80, s = [1,1,1,1])
Activation(LeakyReLU)->Conv(fs = 3,nf=64,s=[1,1,1,1])
Activation(LReLU)->Pool(ps = [1,2,2,1],s=[1,2,2,1]
Conv(fs = 3,nf = 64,s=[1,1,1,1])->Activation(LReLU)
Dropout(prob = 0.75)->Flatten
FullyConnected(output_features = 128)->Dropout(prob = 0.5)
FullyConnected(output_features = 4)


loss_value = tf.reduce_mean(loss_fn)
optimizer = tf.train.AdamOptimizer()

loss_min_fn = optimizer.minimize(loss = loss_value)
check_prediction = tf.equal(tf.argmax(y,axis=1),y_pred)
model_accuracy = tf.reduce_mean(tf.cast(check_prediction, tf.float32)

sess.run(loss_min_fn, feed_dict = {x:X_train_batch, y:y_train_batch})         
train_accuracy = train_accuracy + sess.run(model_accuracy, feed_dict={x : X_train_batch,y:y_train_batch})
train_loss = train_loss + sess.run(loss_value, feed_dict={x : X_train_batch,y:y_train_batch})

The images kind of look like this

Answer 1

I'll make a few observations that will hopefully help.

1. I would remove the dropout layer until you have evidence of overfitting on the data. A dropout layer is generally used to make a model more generalized which is not your problem in this case and it may be hurting your model.
2. The images may not contain enough information to distinguish between the classes. Can an expert assign a class label by looking at the images? If not, it may be that it is just not possible from the images.
3. Your accuracy scores imply that the model is just guessing since a random chance classifier would be expected to perform as well. However, it is curious that you seem to get reasonable accuracy on one class. You may find that you need multiple models configured as one-vs-the-rest binary classifiers. Each model can be uniquely configured and trained to specialize on one class. After the training is complete, which could be time consuming, prediction should be efficient.
4. Lastly, do you think that the model has stopped learning by the time you stop training? Intuitively, I would anticipate the need for tens of thousands of epochs, and potentially millions, before seeing a really refined model. 100 epochs seems like it would be just starting to learn. Because of this cost, I would recommend you configure your model so you can continue training from a stopping point.

And by all means, keep trying different configurations of your layers. One thing to remember though is that each time you use a convolutional layer you are losing information. You need to consider how much information you are losing with each convolutional layer.

Answer 2

I would suggest the following:

Add a 1*1 convolution layer in the beginning of shape 1*1*3. So this will change the image to another colour space if the colour changes are subtle. Add a max pool followed by another 1*1*your_choice after to learn the colour mapping.

Add skip connections(^with this and even otherwise). This will preserve information and speed training.

Your effective receptive field size by the last layer is not enough. Add atleast one more max pool(maybe 2) and two-three conv layers atleast.

It's possible that some of your classes are difficult to distinguish. Use hard mining or focal loss to focus on images of large error. (Maybe decrease drop out and see)

I hope you are pre-processing and normalising the images.

Give us an idea of you dataset and loss curves if you can, also size of images.

Report back.

Answer 3

Late answer, but I was running into a similar issue. I set the learning rate of my Adam optimizer to a lower value (e.g. 3e-5) and voila! The model started fitting.