# LSTM predicting expected value and standard deviation with one-hot encoding

I am implementing LSTM to a time series prediction which requires predicting both the expected value (i.e. the mean of the prediction), and the standard deviation (i.e. the interval that the future value should fall into).

I have considered that one-hot encoding can return the possibility of the prediction of each category, which can then be used to both find the mean and the S.D. of the prediction, for example if the output of the LSTM is

• Category | Possibility
• [0,5) | 0.1
• [5,10) | 0.2
• [10,15) | 0.4
• [15,20) | 0.3

I then conclude that the expected value of the prediction is (2.5x0.1 + 7.5x0.2 + 12.5x0.4 + 17.5x0.3 = 12)

and the S.D. is find in the similar manner.

My question is, is this method a valid method to find the mean and the S.D. of the prediction, why / why not?

The standard deviation and mean of a categorical variable is not meaningful. It looks like the original data are from a range of [0, 20), and the space has been discretized. Now, instead of ranges, you have ordered categories 1 through 4. The individual numbers within each category have lost their meaning. For more details, see the accepted answer here

• Thank you for your answer, can you explain more of how the individual numbers have list their meanings? Dec 28 '17 at 1:54

Expected value and standard deviation are 'independent' in the sense that knowing one gives you no additional information about the other. Therefore it is more accurate and much simpler to train two separate networks - one for the expected value and one for the standard deviation.

This also has the advantage that you need to be explicit about what you mean by 'standard deviation of a (heteroscedastic) time series'.

• Thank you for your answer, actually i have considered training 2 separate networks for mean and s.d. But there is still one problem i face: how do i know the standard deviation of the expected return of past data? (For example, if I collected a training sample of [1, 1.1, 0.9, 1] and the label is [1.3]. I have no idea from which distribution (mean, s.d.) this 1.3 is drawn from. Do you have any solutions for this? Thank you Elias! Feb 26 '18 at 4:40
• What exactly is your setup? I was imagining that you have a time series my_time_series. From this, you calculate a variable my_mean_prediction and another variable my_sd_prediction, each by training an LSTM network. In the end, you can define my_prediction = (my_mean_prediction, my_sd_prediction). Feb 26 '18 at 7:35
• My setup is that i have a time series my_time_series. And then using information from previous time steps [x(t) x(t-1) x(t-2)]. I want to predict x(t+1), but not the value of x(t+1), but the distribution from which x(t+1) is drawn. So to summarize, given [x(t) x(t-1) x(t-2)]. I want to predict the mean and s.d. of the distribution from which x(t+1) is drawn. And i cannot find a way to put the problem into vectors to train my LSTM. Thanks Elias! Feb 26 '18 at 15:46
• Have you managed to train the LSTM that predicts the expected value? That is a standard problem and you will find many tutorials and articles by googling 'time series forecasting'. Feb 26 '18 at 15:59
• Predicting standard deviation is less standard, but please be aware that the main problem is figuring out what you actually want to compute ;) There are different approaches to estimating variance/standard deviation in a heteroscedastic time series. A simple approach might be to define $my_time_series_2 = my_time_series ** 2$ and forecast that as well. That would give you an estimate for the (uncentered) second moment. Feb 26 '18 at 16:01