Before marking my question as duplicate, I would like to say that I have tried all the possible solutions mentioned in similar questions, but that doesn't seem to work.
I am currently working on blood cells classification problem where we basically have to classify blood images (4 classes). The dataset consists of 9957 images, nearly equal number of images of all the 4 classes. The accuracy always hovers around 25-27% even after trying different optimizers and learning rates. I even tried training upto 100 epochs. Image augmentation doesn't help. Also, it is not that it is predicting same class for all images although for 1 particular batch of images, it predicts the same class. It again predicts some other class for all images in the next batch. So, I would just like to know, what am I possibly doing wrong? Is the dataset not sufficient, or the architecture should have more hidden layers, or am I not implementing optimizer or loss function correctly or is there any silly mistake I am overlooking in my code ?
My CNN architecture: (fs means filter_size, nf means number of filters, s is no. strides)
Input(80,80,1)->Conv(fs = 3, nf = 80, s = [1,1,1,1])
Activation(LeakyReLU)->Conv(fs = 3,nf=64,s=[1,1,1,1])
Activation(LReLU)->Pool(ps = [1,2,2,1],s=[1,2,2,1]
Conv(fs = 3,nf = 64,s=[1,1,1,1])->Activation(LReLU)
Dropout(prob = 0.75)->Flatten
FullyConnected(output_features = 128)->Dropout(prob = 0.5)
FullyConnected(output_features = 4)
loss_value = tf.reduce_mean(loss_fn)
optimizer = tf.train.AdamOptimizer()
loss_min_fn = optimizer.minimize(loss = loss_value)
check_prediction = tf.equal(tf.argmax(y,axis=1),y_pred)
model_accuracy = tf.reduce_mean(tf.cast(check_prediction, tf.float32)
sess.run(loss_min_fn, feed_dict = {x:X_train_batch, y:y_train_batch})
train_accuracy = train_accuracy + sess.run(model_accuracy, feed_dict={x : X_train_batch,y:y_train_batch})
train_loss = train_loss + sess.run(loss_value, feed_dict={x : X_train_batch,y:y_train_batch})