2
$\begingroup$

I am implementing LSTM to a time series prediction which requires predicting both the expected value (i.e. the mean of the prediction), and the standard deviation (i.e. the interval that the future value should fall into).

I have considered that one-hot encoding can return the possibility of the prediction of each category, which can then be used to both find the mean and the S.D. of the prediction, for example if the output of the LSTM is

  • Category | Possibility
  • [0,5) | 0.1
  • [5,10) | 0.2
  • [10,15) | 0.4
  • [15,20) | 0.3

I then conclude that the expected value of the prediction is (2.5x0.1 + 7.5x0.2 + 12.5x0.4 + 17.5x0.3 = 12)

and the S.D. is find in the similar manner.

My question is, is this method a valid method to find the mean and the S.D. of the prediction, why / why not?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
2
$\begingroup$

The standard deviation and mean of a categorical variable is not meaningful. It looks like the original data are from a range of [0, 20), and the space has been discretized. Now, instead of ranges, you have ordered categories 1 through 4. The individual numbers within each category have lost their meaning. For more details, see the accepted answer here

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you for your answer, can you explain more of how the individual numbers have list their meanings? $\endgroup$
    – Derpson
    Dec 28, 2017 at 1:54
0
$\begingroup$

Expected value and standard deviation are 'independent' in the sense that knowing one gives you no additional information about the other. Therefore it is more accurate and much simpler to train two separate networks - one for the expected value and one for the standard deviation.

This also has the advantage that you need to be explicit about what you mean by 'standard deviation of a (heteroscedastic) time series'.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Thank you for your answer, actually i have considered training 2 separate networks for mean and s.d. But there is still one problem i face: how do i know the standard deviation of the expected return of past data? (For example, if I collected a training sample of [1, 1.1, 0.9, 1] and the label is [1.3]. I have no idea from which distribution (mean, s.d.) this 1.3 is drawn from. Do you have any solutions for this? Thank you Elias! $\endgroup$
    – Derpson
    Feb 26, 2018 at 4:40
  • $\begingroup$ What exactly is your setup? I was imagining that you have a time series my_time_series. From this, you calculate a variable my_mean_prediction and another variable my_sd_prediction, each by training an LSTM network. In the end, you can define my_prediction = (my_mean_prediction, my_sd_prediction). $\endgroup$
    – Elias Strehle
    Feb 26, 2018 at 7:35
  • $\begingroup$ My setup is that i have a time series my_time_series. And then using information from previous time steps [x(t) x(t-1) x(t-2)]. I want to predict x(t+1), but not the value of x(t+1), but the distribution from which x(t+1) is drawn. So to summarize, given [x(t) x(t-1) x(t-2)]. I want to predict the mean and s.d. of the distribution from which x(t+1) is drawn. And i cannot find a way to put the problem into vectors to train my LSTM. Thanks Elias! $\endgroup$
    – Derpson
    Feb 26, 2018 at 15:46
  • $\begingroup$ Have you managed to train the LSTM that predicts the expected value? That is a standard problem and you will find many tutorials and articles by googling 'time series forecasting'. $\endgroup$
    – Elias Strehle
    Feb 26, 2018 at 15:59
  • $\begingroup$ Predicting standard deviation is less standard, but please be aware that the main problem is figuring out what you actually want to compute ;) There are different approaches to estimating variance/standard deviation in a heteroscedastic time series. A simple approach might be to define $my_time_series_2 = my_time_series ** 2$ and forecast that as well. That would give you an estimate for the (uncentered) second moment. $\endgroup$
    – Elias Strehle
    Feb 26, 2018 at 16:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.