2
$\begingroup$

I am trying to construct a machine learning model that predicts the difference in price from tomorrow to the day after tomorrow, using yesterday's OHLCV (open, high, low, close, volume).

My models (LSTM and DNNs), however, give me very poor outputs. I think it is because the data needs to be reformatted in some way, shape, or form (i.e. normalized, log-pct change, etc.)

What is the best way to manipulate the input? Should I use a sci-kit 'scaler', or any other method to normalize the data? If so, what range should the scaler be within?

What about the output? If I want the output to be linear (continuous, non-softmax value), should I manipulate the output data as well to be scaled, and then unscale the predicted output to match the original labels? Does it matter?

Improve this question
$\endgroup$
4
  • $\begingroup$ Is the data stationary? $\endgroup$
    – Hobbes
    Jul 26, 2017 at 15:49
  • $\begingroup$ Well, not at the moment. Would that be something you'd advise doing? The data is pulled straight out of Google. $\endgroup$
    – Landmaster
    Jul 27, 2017 at 0:12
  • $\begingroup$ Predicting stock prices from their own dynamic and nothing else is very likely to give poor results $\endgroup$
    – Neil Slater
    Jul 27, 2017 at 6:20
  • $\begingroup$ Haha, that's ok! I just want to see something work out to the best of its ability. $\endgroup$
    – Landmaster
    Jul 28, 2017 at 4:20

1 Answer 1

Reset to default
1
$\begingroup$

I used a MinMax scaler in the range between (0, 1) applied to the closing price of S&P500. The RNN consisted of a single LSTM layer with a lookback window of 10 days to predict the next day's closing price. The following figure shows RNN prediction of the next day's closing price (in red).

enter image description here

For code used to generate the figure, have a look at the following ipython notebook.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This looks great! However, be aware that when you use MinMax scaler and fit it to the entire dataset, you're unfortunately looking ahead into your test data in order to establish the range, which makes the back test leak data slightly. $\endgroup$
    – Landmaster
    Aug 16, 2017 at 0:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.