1
$\begingroup$

I am writing my masters thesis and am using LSTMs for daily stock return prediction. So far I am only predicting numerical values but will soon explore a classification style problem and predict whether it will go up or down each day.

I have explored several scenarios

  • A single LSTM using as input only the past 50 days return data
  • A stacked (2 layers) using as input only the past 50 days return data

The results are not great for either (and I didn't expect them to be). So I tried some feature engineering using 3 day MA, 5 day MA, 10 day MA, 25 day MA, 50 day MA of the daily returns as well as the actual daily return, meaning I have 6 input features. All other variables are kept constant yet the model now overfits (see the training and test loss plots below). Does anyone have any ideas why this may be?

Test Loss in orange and Train in blue Test Loss in orange and Train in blue

$\endgroup$

2 Answers 2

2
$\begingroup$

I am not sure this type of model is a good use case for the particular task. More specifically, citing Chollet from Deep Learning with Python book,

Always remember that when it comes to markets, past performance is not a good predictor of future returns—looking in the rear-view mirror is a bad way to drive. Machine learning, on the other hand, is applicable to datasets where the past is a good predictor of the future.

— Deep learning with python, Francois Chollet

What is essentially being argued here is that, stock historical data is not a phenomenon that repeats itself based on its own underlying distribution.

$\endgroup$
5
  • 1
    $\begingroup$ Thanks for answering. There are several papers that have done seemingly decent results with this, including this paper: cs230.stanford.edu/projects_winter_2020/reports/32066186.pdf, I was more interested in understanding why Feature Engineering has appeared to cause overfitting? Feature Engineering (and hence adding more data should do the opposite) $\endgroup$ Apr 26, 2021 at 14:01
  • $\begingroup$ That's a very interesting read. What I am essentially arguing is that stock historical data is just not a phenomenon that repeats itself based on its own underlying distribution. $\endgroup$
    – hH1sG0n3
    Apr 26, 2021 at 15:29
  • $\begingroup$ I do agree with you entirely but its what I have to work with so I have to give it my best shot. It is curious that I got those results $\endgroup$ Apr 27, 2021 at 8:03
  • $\begingroup$ I hear you, and my answer does not answer any of your particular question to be honest. Would you mind sharing your code so we can maybe look more specifically into those? $\endgroup$
    – hH1sG0n3
    Apr 27, 2021 at 11:01
  • $\begingroup$ Sure - what would be the most appropriate way? Should I post it here? I am limited in what I can share with regards to the data as that falls under the licensing agreement between myself and the industry partner $\endgroup$ Apr 27, 2021 at 21:23
0
$\begingroup$

First of all, adding features to your model can easily cause overfitting. Even adding a column of random values can cause overfitting. Take for example your series with exactly one feature - the price. The price can be the same value X several times - first when the series is rising, and then when it's decreasing. if you only use the price, you are forced to try to learn the average of the future value at that price.

Now add a moving average. Now when the price is X and the series is rising, the MA will be less than X, while when the series is decreasing the MA will be more than X. your model can now on one hand learn to predict better (if the MA is actually a good feature), but it can also hack the data batter because it is now better able to memorize the training set.

Now add another 50 moving averages to the features. you may now have a unique value of features for every output, so your model can fit the data extremely well, but obviously, that noise will not predict anything on the test set.

How many features of (basically) the same thing to use is a good question, but you need to remember you're not actually adding a lot of information with those features. And frankly, LSTMs can calculate moving averages pretty well themselves.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.