I am trying to construct a machine learning model that predicts the difference in price from tomorrow to the day after tomorrow, using yesterday's OHLCV (open, high, low, close, volume).

My models (LSTM and DNNs), however, give me very poor outputs. I think it is because the data needs to be reformatted in some way, shape, or form (i.e. normalized, log-pct change, etc.)

What is the best way to manipulate the input? Should I use a sci-kit 'scaler', or any other method to normalize the data? If so, what range should the scaler be within?

What about the output? If I want the output to be linear (continuous, non-softmax value), should I manipulate the output data as well to be scaled, and then unscale the predicted output to match the original labels? Does it matter?

  • $\begingroup$ Is the data stationary? $\endgroup$ – Hobbes Jul 26 '17 at 15:49
  • $\begingroup$ Well, not at the moment. Would that be something you'd advise doing? The data is pulled straight out of Google. $\endgroup$ – Landmaster Jul 27 '17 at 0:12
  • $\begingroup$ Predicting stock prices from their own dynamic and nothing else is very likely to give poor results $\endgroup$ – Neil Slater Jul 27 '17 at 6:20
  • $\begingroup$ Haha, that's ok! I just want to see something work out to the best of its ability. $\endgroup$ – Landmaster Jul 28 '17 at 4:20

I used a MinMax scaler in the range between (0, 1) applied to the closing price of S&P500. The RNN consisted of a single LSTM layer with a lookback window of 10 days to predict the next day's closing price. The following figure shows RNN prediction of the next day's closing price (in red).

enter image description here

For code used to generate the figure, have a look at the following ipython notebook.

  • 1
    $\begingroup$ This looks great! However, be aware that when you use MinMax scaler and fit it to the entire dataset, you're unfortunately looking ahead into your test data in order to establish the range, which makes the back test leak data slightly. $\endgroup$ – Landmaster Aug 16 '17 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.