# Is "adding the predictions to the real data for new training and prediction" a good idea for LSTM?

Considering we have trained our model with a lot of data for "many-to-one" prediction. Then we like to forecast the future data of next 10 days. So we use last 60 of existent data and predict the single next day. From here there are 2 approaches:

1. We can put our model.predict() function in a for loop for 10 times and do predictions like this(adding our predictions to end of our real data).

2. We can put all of our model(consisting training part, not just predict part), in a for loop and this means we train our model 10 times whenever we do a new prediction and adding it to our real data.

EDIT: Thinking you have X_train = (100,60,1) array that means 100 examples, 60 time-steps(hidden units) and 1 feature for each example. Also you have y_train array of size (100,1,1) that means 100 labels with time-steps = 1 and 1 feature. Then you train your network to read 60 of inputs and predict the next single output. Also you create a X_test array like this: X_test = X_train[len(X_train - 60):] that means you use last 60 numbers of your series to predict the next number. So you use the new_number = model.predict(X_test) for that and you predict the time-step 61 that is not a real number. It's your prediction. Then you want to continue your predictions. So what do you do is adding the 61'th predicted number to the last of your X_test = np.append(X_test, new_number) and do new number = model.predict(X_test) again. But the difference is that the last number in your new X_test is your previous prediction. And you keep this way for 10 times to predict 10 next numbers. (This was the first approach).

The other approach(2) has a difference. After doingnew_number = model.predict(X_test) for the first time, you add the predicted number to x_train instead of X_test, like this X_train = np.append(X_train, new_number) and train your model again model.fit(X_train , y_train) with the new predicted number. Then you use new number = model.predict(X_test) and again adding predicted number into the X_train, then train your model again(this time, with 2 new predicted numbers that you have added to the end of your X_train) and so on for 10 times!

• Using TimeDistributed() wrapper of Keras, you will be able to predict upcoming 10 at once. Have a look at machinelearningmastery.com/… and quora.com/What-is-time-distributed-dense-layer-in-Keras Feb 26 '19 at 17:24
• @UgurMULUK: I have edited my question and tried to explain more in EDIT section. Please read it. Maybe it would help to clear what I ask.
– WDR
Feb 26 '19 at 18:03
• Is many-to-one prediction is mandatory for you? The reason I ask is that LSTM passes the predictions of each recurrent neuron to the next one. If you can train your network as many-to-10 by using the labels of next 10 with previous 60, your LSTM model will automatically predict the next 10 prediction by predicting each of the days from forward by using the predictions of the previous future prediction. Have a look at the first image here: karpathy.github.io/2015/05/21/rnn-effectiveness The 4th RNN structure in that image is what I think as would work for your case. Feb 27 '19 at 19:41
• @UgurMULUK: You mean LSTM automatically does what I explained in approach 2?
– WDR
Feb 27 '19 at 19:57
• Yes I mean that, LSTMs (RNNs in a general manner) has the structure of using previous prediction in the next recurrent layer by nature; if you structure your network properly (which can be done in Keras), you can have what you expect here. Have a look at this video, especially at the end part where there are different RNN structures: youtube.com/… Feb 27 '19 at 23:41

Are you saying you would like to predict 10 days ahead in this instance?

If this is the case, the LSTM model is able to do this and iterating in the way you are suggesting is unnecessary and could give you unreliable results.

For instance, consider a dataset whereby we are attempting to predict one-step ahead:

# Training and Test data partition
train_size = int(len(dataset) * 0.8)
test_size = len(dataset) - train_size
train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]

# reshape into X=t and Y=t+1
previous = 1
X_train, Y_train = create_dataset(train, previous)
X_test, Y_test = create_dataset(test, previous)

# reshape input to be [samples, time steps, features]
X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))

# Generate LSTM network
model = Sequential()
model.fit(X_train, Y_train, epochs=100, batch_size=1, verbose=2)

# Generate predictions
trainpred = model.predict(X_train)
testpred = model.predict(X_test)

# Convert predictions back to normal values
trainpred = scaler.inverse_transform(trainpred)
Y_train = scaler.inverse_transform([Y_train])
testpred = scaler.inverse_transform(testpred)
Y_test = scaler.inverse_transform([Y_test])

# calculate RMSE
trainScore = math.sqrt(mean_squared_error(Y_train[0], trainpred[:,0]))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = math.sqrt(mean_squared_error(Y_test[0], testpred[:,0]))
print('Test Score: %.2f RMSE' % (testScore))


In this batch of code, you can see that we have set the previous parameter equal to 1, meaning that the time step being considered by the model is t-1.

In this particular instance, here are the training and test predictions compared to the actual series:

Now, the same model is run, but this time the previous parameter is set to 10. In other words, the previous 10 days are being considered as one time step, and the model is forecasting for time t+10 in this instance. Here is another sample prediction comparing the test set with the actual. A fuller example of this is provided here:

In this regard, my advice would be to define the time series you wish to forecast and then work off that basis. Using iterations only complicates the situation, and could even cause issues with prediction.

• I have edited my question and tried to explain more in EDIT section. Please read it. Maybe it would help to clear what I ask.
– WDR
Feb 26 '19 at 18:02