I have a binary classification project, I use a neural network with the following architecture:
The shape of the input is 64×64×4. This input was fed to a Conv2D layer with 32(5×5) filters followed by a max pool layer of 2×2 and the output of this layer was fed to another Conv2D layer with 32(3x3) filters followed by a max pool again, then a dropout layer is then applied to avoid overfitting. The output of the dropout layer is then flattened. The flattened data was then sent to two LSTM layers, after which self-attention was applied to the encoded data from LSTM. The last layer was a dense/fully connected (FC) layer. The output of FC was passed through a softmax layer to get the prediction i.e. sabotage or no-sabotage. Batch Normalization was applied in training to normalize the outputs of the layer.
My data is time series data. The data has 4 columns (4 channels) divided into 12 subjects. After doing some preprocessing and converting the data to spectrograms, Each subject has around 2-3K data points (after preprocessing) of shape (N, 64, 64, 4) for each class. When I run my model on intra-subject data (i.e. train with data for subject 1 and test on subject 1) I get an average validation accuracy of 98% and test accuracy of 93%.
But when I run my model on inter-subject data (i.e. train on 11 random subjects and test on the subject that is left out) I get an average validation accuracy of 45-55%% and test accuracy of 50-60%.
I have a few Questions:
- How adding more data can hurt the accuracy of my model?
- Based on my current architecture what can I change to increase my accuracy (at least 80%)?