# CNN not learning properly

Before marking my question as duplicate, I would like to say that I have tried all the possible solutions mentioned in similar questions, but that doesn't seem to work.

I am currently working on blood cells classification problem where we basically have to classify blood images (4 classes). The dataset consists of 9957 images, nearly equal number of images of all the 4 classes. The accuracy always hovers around 25-27% even after trying different optimizers and learning rates. I even tried training upto 100 epochs. Image augmentation doesn't help. Also, it is not that it is predicting same class for all images although for 1 particular batch of images, it predicts the same class. It again predicts some other class for all images in the next batch. So, I would just like to know, what am I possibly doing wrong? Is the dataset not sufficient, or the architecture should have more hidden layers, or am I not implementing optimizer or loss function correctly or is there any silly mistake I am overlooking in my code ?

My CNN architecture: (fs means filter_size, nf means number of filters, s is no. strides)

Input(80,80,1)->Conv(fs = 3, nf = 80, s = [1,1,1,1])
Activation(LeakyReLU)->Conv(fs = 3,nf=64,s=[1,1,1,1])
Activation(LReLU)->Pool(ps = [1,2,2,1],s=[1,2,2,1]
Conv(fs = 3,nf = 64,s=[1,1,1,1])->Activation(LReLU)
Dropout(prob = 0.75)->Flatten
FullyConnected(output_features = 128)->Dropout(prob = 0.5)
FullyConnected(output_features = 4)

loss_value = tf.reduce_mean(loss_fn)

loss_min_fn = optimizer.minimize(loss = loss_value)
check_prediction = tf.equal(tf.argmax(y,axis=1),y_pred)
model_accuracy = tf.reduce_mean(tf.cast(check_prediction, tf.float32)

sess.run(loss_min_fn, feed_dict = {x:X_train_batch, y:y_train_batch})
train_accuracy = train_accuracy + sess.run(model_accuracy, feed_dict={x : X_train_batch,y:y_train_batch})
train_loss = train_loss + sess.run(loss_value, feed_dict={x : X_train_batch,y:y_train_batch})


The images kind of look like this

• Add more convolutional layers, the deeper you go the more number of filters you need. Also try to add an extra fully connected layer. As optimiser, try to exploit Adam. Nov 8, 2018 at 13:16

I'll make a few observations that will hopefully help.

1. I would remove the dropout layer until you have evidence of overfitting on the data. A dropout layer is generally used to make a model more generalized which is not your problem in this case and it may be hurting your model.
2. The images may not contain enough information to distinguish between the classes. Can an expert assign a class label by looking at the images? If not, it may be that it is just not possible from the images.
3. Your accuracy scores imply that the model is just guessing since a random chance classifier would be expected to perform as well. However, it is curious that you seem to get reasonable accuracy on one class. You may find that you need multiple models configured as one-vs-the-rest binary classifiers. Each model can be uniquely configured and trained to specialize on one class. After the training is complete, which could be time consuming, prediction should be efficient.
4. Lastly, do you think that the model has stopped learning by the time you stop training? Intuitively, I would anticipate the need for tens of thousands of epochs, and potentially millions, before seeing a really refined model. 100 epochs seems like it would be just starting to learn. Because of this cost, I would recommend you configure your model so you can continue training from a stopping point.

And by all means, keep trying different configurations of your layers. One thing to remember though is that each time you use a convolutional layer you are losing information. You need to consider how much information you are losing with each convolutional layer.

• Thanks for giving a quick response. As you suggested, I removed dropout (set probability as 1). Tried different configs (added more fully connected layer, stacks of convolutional and pooling layer) but still the accuracy remains the same. I have added images to give an idea of dataset. The problem is that it is giving nearly equal probability score for all the 4 classes. Nov 9, 2018 at 3:57
• Assuming, since you have four images, one is from each of the classes. It seems like there is enough visual information to distinguish the classes. I am wondering if you are processing these images or do you do anything before you train? Given your performance, I'd be tempted to try crop the image to, for example, just the range of the purple + a margin, or just colors in a specific range. Maybe there is to much noise in the image? You said you tried augmentation, so maybe you've tried these already. Did you try the one-vs-the-rest approach? Nov 9, 2018 at 12:43
• Could you please suggest me preprocessing steps that I could implement to achieve a decent result with these images ? Nov 12, 2018 at 3:49

I would suggest the following:

Add a 1*1 convolution layer in the beginning of shape 1*1*3. So this will change the image to another colour space if the colour changes are subtle. Add a max pool followed by another 1*1*your_choice after to learn the colour mapping.

Add skip connections(^with this and even otherwise). This will preserve information and speed training.

Your effective receptive field size by the last layer is not enough. Add atleast one more max pool(maybe 2) and two-three conv layers atleast.

It's possible that some of your classes are difficult to distinguish. Use hard mining or focal loss to focus on images of large error. (Maybe decrease drop out and see)

I hope you are pre-processing and normalising the images.

Give us an idea of you dataset and loss curves if you can, also size of images.

Report back.

• 1*1 convolutional layer means convolutional layer of kernel size [batch_size,1,1,3] ? Nov 9, 2018 at 3:47
• By the way, color is not an important feature in our dataset so I converted the images to grayscale images. I am adding random rotation, random flip up down and random flip left right as part of image augmentation. I have added images to give you an idea of my dataset. The images are originally of size 240x320, but for my model, I resized them to 80x80. Nov 9, 2018 at 3:50
• have you figured it out? Nov 10, 2018 at 19:35
• No, I still didn't figure it out. What preprocessing steps would you suggest to solve this problem ? Nov 12, 2018 at 3:46

Late answer, but I was running into a similar issue. I set the learning rate of my Adam optimizer to a lower value (e.g. 3e-5) and voila! The model started fitting.