Improve test accuracy for TensorFlow CNN

Question

I'm trying to use Tensorflow for signal classification. The signals are either normal or high-risk signals. For this purpose, I used convolutional neural networks. The length of signals are 685 and the architecture is:

• Convolution layer with 27 channels and 1 by 16 window size and stride 1.
• Max pooling layer with 1 by 2 window size and stride 2.
• Convolution layer with 14 channels and 1 by 32 window size and stride 1.
• Max pooling layer with 1 by 2 window size and stride 2.
• Convolution layer with 4 channels and 1 by 32 window size and stride 1.
• Max pooling layer with 1 by 2 window size and stride 2.
• Convolution layer with 3 channels and 1 by 10 window size and stride 1.
• Max pooling layer with 1 by 2 window size and stride 2.
• Fully connected layer with 20 neurons and dropout layer.
• Fully connected layer with 10 neurons and dropout layer.
• And finally Soft max layer.

After training the network using AdamOptimizer with learning rate 0.001 with 150000 signals the training accuracy is near 95 percent (batch training with 16 batch size is used), however testing accuracy using 20000 new signals is almost 50 percent. Since there is just 2 classes, this accuracy is no better than random guess.

model="conv1d-27-16-1,maxpool-2,conv1d-14-32-1,maxpool-2,conv1d-4-32-1,maxpool-2,conv1d-3-10-1,maxpool-2,full-20,full-10,softmax"

cnn=CNN_1D(model,input_size=685,n_classes=2,num_epochs=20,batch_size=16,dropout=0.75)
cnn.read_data('train_input','train_targets','test_input','test_targets')
cnn.build_model()
cnn.training(validation_set='all')

How can I improve the testing accuracy in my network?

Answer 1

This is clearly a case of overfitting, since your validation error is way bigger than your training error. Having understood that overfitting is your problem, there are many ways you can address this problem:

• Dropout (keras)
• Weight decay (L2 regularization)
• Early stopping (keras)

I can see that you are already doing dropout, so setting the dropout higher should help. Moreover, you can try using dropout in more layers, not just the two last ones. If setting the dropout rate higher and setting more layers with dropout does not work, then this probably means that your training and validation data do not come from the same distribution. You can study statistical properties of your training data and validation and study if they come from the same distribution.

Extra comment: if there are just two classes, why don't you use a sigmoid activation function in the last layer?