Skip to main content
added 343 characters in body
Source Link
Louis T
  • 1.1k
  • 8
  • 22

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chanceIf you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.


Correction. Had a bit better thinking about it. it is you are having so-called "gradientnot a gradient explosion problem", tryproblem here. More likely it is due to clip the gradient before backprop might besaturated neuron. Because LSTM uses sigmoid function, if the solutioninput is not properly scaled or weight is initialised inproperly, it could cause neuron saturation very quickly.

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.


Correction. Had a bit better thinking about it. it is not a gradient explosion problem here. More likely it is due to saturated neuron. Because LSTM uses sigmoid function, if the input is not properly scaled or weight is initialised inproperly, it could cause neuron saturation very quickly.

added 2 characters in body
Source Link
Louis T
  • 1.1k
  • 8
  • 22

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.

Source Link
Louis T
  • 1.1k
  • 8
  • 22

Is this the correct way to forecast time series with LSTMs?

Train data independent matrix (X)=Sequences of previous 30 day values Train (Y)=The 31st day value for each of previous 30 day values

You method could work in theory. But typically, for vanilla LSTM, the setup looks like $x = [x_0, x_1, ..., x_{n-1}]$ and $y = [x_1, x_2 ..., x_n]$. I suspect you will get much better result if you do it this way.

Highly recommend Colah's excellent explaination if you haven't read it already.

I followed the following methodology to forecast: Y for t+1 is first forecasted, Then X matrix row is shifted by 1 day and the forecasted Y is appended to the end of this row, then use this row to predict t+2 value and continue. However in each sequence after (usually) t+3 days the forecasted values become constant for the rest of the t+n days.

This is not recommended because any error in prediction is compounded over time. please find my answer in the following post for a detailed explanation Is time series multi-step ahead forecasting a sequence to sequence problem?

How can this behaviour be explained?

If you are getting constant prediction value, the chance is you are having so-called "gradient explosion problem", try to clip the gradient before backprop might be the solution.