LSTM model bad forecasts

Question

I tried to implement LSTM model for time-series prediction. Below is my trial code. This code runs without error.

metrics = ['mean_squared_error', 'mean_absolute_error', 'mean_absolute_percentage_error']


# define model
model = Sequential()
model.add(LSTM(100, activation='relu', input_shape=(n_past, 6), return_sequences=True))
model.add(LSTM(100, activation='relu', return_sequences=True))
model.add(LSTM(50, activation='relu'))
model.add(Dense(2, activation='relu'))
model.compile(optimizer='adam', loss='mse', metrics=metrics)
model.summary()

)

The forecast correctly predicts the peaks, but the constant values ​​should be 0 and you cannot predict it.

However, prediction is extremely poor. How to improve the predictin? Do you have any ideas to improve it? Any ideas for improving prediction by re-designing architecture and/or layers?

Answer 1

Your loss function estimates the conditional expectation and the predictions mainly hover around the mean of the time series. I would argue that predictions are good, but the problem is poorly designed. One approach would be to partition the problem into two subproblems: probability of your series being $\mathbb{P}(Y>0)$ and predicting the conditional expectation by training only on non-zero values $\mathbb{E}[Y|Y>0]$. The final prediction would then be $\mathbb{P}(Y>0)\mathbb{E}[Y|Y>0]$