9
$\begingroup$

We need to implement a time series problem with the LSTM model.

But, while implementing the same, the main challenge I am facing is the feature selection issue. Because our data-set contains 2300 observations and 600 features. And we already know there are so many features which are not at all relevant. But, no one has the domain expertise to confirm which feature is relevant.

I have tried with…

  1. All 600 features, [shape of X become, 2280 X 20 X 600 (if time step is 20)]. The accuracy is very poor 53-55%.

  2. Considered a single column as a predictor (the feature which we want to predict) [shape of X become, 2280 X 20 X 1 (if the time step is 20)]. The accuracy is also poor.

  3. Also, tried with PCA so that all 600 features are reduced to 20 features [approx 98% data was retained], but not seen any improvement.

  4. Finally, I have gone through a very lengthy process.

    a. Out of 600 features, each of them is tried out individually with the LSTM algorithm. The feature which was giving the best result is selected.

    b. The selected feature is paired with the rest of 599 features individually. Then, I select the best pair that gives the best output.

    c. The best pair is further tried with rest 598 features individually and this process continues...

After doing so, as of now, we get 6 features, the accuracy gets improved to 65%.

My question is, is it the only approach that I can do feature selection in time series problems?

In case of any non-time series problem there are so many ways to do the feature selection. What will be the best way to do feature selection in the time series issue?

Let's give another example...

Let's assume we developed the LSTM model with 10 predictors/features. The model is working very fine and we get 80% accuracy.

Now, somehow 100 more features get added which are not relevant for deriving target y.

So, the accuracy will be dropped as the model will focus on 120 predictors/features, where 100 are not relevant.

Then, how will we eliminate the 100 unwanted features?

Improve this question
$\endgroup$
1
  • $\begingroup$ I thought LSTMs are used for time series forecasting, a kind of a regression problem. So, I wonder why you use as a metric accuracy in % (as in classification problems) instead of loss-value or MSE? Probably I am getting something wrong $\endgroup$
    – NeStack
    Jan 31, 2021 at 18:16

2 Answers 2

Reset to default
3
$\begingroup$

Have you tried a correlation test between the output and the features used to model it?

I usually use Pearson's Correlation to determine the features with low correlation factors. Then I remove them from the training.

Here, I have DOC_DS used to model DOC_US; I have them both in a data frame df with my other features. From the trials I did on my models, the features with less correlation affected the model accuracy later on.

In Python Environment:

print(np.array_str(np.corrcoef(df.DOC_DS, df.DOC_US)))
Improve this answer
$\endgroup$
3
  • 7
    $\begingroup$ Keep in mind that correlation shows latent linear relations. In the case of an LSTM and generally in the case of any NN with more than 1 layer, the point is to model nonlinear functions and relations. I would be extremely reluctant to trust correlation coefficients to reduce the dimensionality of my nonlinear model. Keep in mind: "correlation does not imply causation". $\endgroup$
    – pcko1
    May 28, 2018 at 16:48
  • $\begingroup$ correlation is not a good metric for non-linear relations $\endgroup$
    – Remis Haroon - رامز
    Jun 28, 2020 at 14:15
  • $\begingroup$ Apart from what people mentioned about correlation doesn't matter in this case, the author has tried PCA which eliminates correlation but didn't works in the 2nd approach. $\endgroup$
    – SKSKSKSK
    Feb 4, 2022 at 0:41
1
$\begingroup$

Check out the shap library. I think that could help you.

https://github.com/slundberg/shap

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.