I am training a CNN with 2 conv layers 2 Relu and max pooling and 2 FC layers the last of which has only 2 units since it's a binary classification problem. The images are spatio-temporal continuous, basically those of stream functions of the atmosphere, obtained as contour plots in MATLAB from data generated out of a PDE solver. So, it is quite difficult to understand the difference between the two labels by just looking at the respective images. The labels are generated through some statistical algorithm that operates on the intensity of the pixels or simply the value of the matrices that make the contour plot. While training the CNN, I see that with a learning rate of .001, the loss decreases gradually and monotonically at all time where it goes down to 0.6 in the first 200 epochs (not suddenly, quite gradually, the slope decreasing as the value goes down) and settles there for the next 500 epochs. However, the training accuracy fluctuates around 50% and my testing accuracy is 50% too. Moreover, I see while looking at the false +ve and -ves that the CNN has got one label right all the time and the other label wrong all the time. I have used dropout of 0.5 and L2 regularization. I am using TensorFlow for all the programming. There is no imbalance in the training set between the TRUE and FALSE labels. I have 4500 TRUE example and 4500 False examples. I have randomly shuffled the training set and normalized them with 255 since I'm using RGB images. Testing is done on 200 TRUE examples and 200 False examples. Kernel size 5x5 in both layers , first one having 32 and second one having 64 filters and FC layer has 1024 neurons. I have played around with these parameters with no real improvement. Any suggestions would be really appreciated.
1 Answer
I guess I was able to figure out the answer to this when I looked carefully at the feature space and printed out the outputs from different convolutional layers. It appears that this issue appears when you have a lot of redundant features in your feature space. Since, this was a binary classification problem with FALSE and TRUE, each image by default had a lot of features that would correspond to a FALSE and the TRUEs only had a small percentage of the feature space being different. The trainer easily learnt the large percentage of the FALSE feature space and hence the loss easily went down while the small TRUE feature space was missed. This is the reason why the loss was going down but accuracy not improving. The trick to avoiding this issue was to downsample the initial image in the beginning before going to the convolutional layer. One could do it with some pyramidal pooling if on Tensorflow, or simply with imresize in matlab.