I am trying to train an LSTM(many to one) model with multivariate time series input and a categorical output.
after training for quite some time, the resulting model still has low accuracy and high loss on validation data. So started doubting that maybe the features I had chosen for predicting the label are simply irrelevant?
I wonder if there are methods for testing if the chosen features do have explanatory power on labels?
Before coming to stack exchange, I did some research online and found somebody saying we should be using PCA to test if labels are dependent on features, which confused me a lot. I thought PCA is used for dimension reduction on features and is irrelevant to labels. am I missing something here?
Below is an extract of what I just mentioned.
- It is always a good idea first to make sure that the output (dependent) variable (target or label) actually depends on the input variables (features). It is possible that you are chasing a ghost that doesn’t exist. There is a way to check this, but before that, we have step two.
- Start by using the z-scores to normalize the input variables. Any normalizing would do but there is a reason for using z-scores. It has to do with the next step. You can do a Principal Component Analysis (PCA). It will tell you the contribution of each of the new variables (obtained after the transformation) to the variation on the output variable.
- PCA will answer the question I mentioned at the outset about the existence of dependency clearly. Before performing PCA, the variables have to be normalized using z-scores.
