LSTMs how to forecast out N steps

Question

I have about 3 weeks of 15 minute building electricity power data and curious to know how can I predict an entire days worth of electricity into the future? 96 Future values that makes up 24 hours...my model only outputs/predicts 1 step into the future any tips greatly appreciated how to modify my approach.

I can make this LSTM model where I am only using 2 EPOCHS just for testing purposes:

# https://machinelearningmastery.com/time-series-prediction-lstm-recurrent-neural-networks-python-keras/

import numpy as np
import matplotlib.pyplot as plt
import tensorflow as tf
from pandas import read_csv
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import mean_squared_error

power = read_csv('https://raw.githubusercontent.com/bbartling/Data/master/months_data.csv', index_col=[0], parse_dates=True)

plt.figure(figsize=(15, 7))
plt.plot(power.kW)
plt.title('kW 15 Minute Intervals')
plt.grid(True)
plt.show()

def create_dataset(dataset, look_back=1):
    dataX, dataY = [], []
    for i in range(len(dataset)-look_back-1):
        a = dataset[i:(i+look_back), 0]
        dataX.append(a)
        dataY.append(dataset[i + look_back, 0])
    return np.array(dataX), np.array(dataY)

# normalize the dataset
scaler = MinMaxScaler(feature_range=(0, 1))
dataset = scaler.fit_transform(power.values)

EPOCHS = 2 # just for testing purposes

# split into train and test sets
train_size = int(len(dataset) * 0.67)
test_size = len(dataset) - train_size
train, test = dataset[0:train_size,:], dataset[train_size:len(dataset),:]

# reshape into X=t and Y=t+1
look_back = 96 # number of 15 min intervals in one day

trainX, trainY = create_dataset(train, look_back)
testX, testY = create_dataset(test, look_back)

# reshape input to be [samples, time steps, features]
trainX = np.reshape(trainX, (trainX.shape[0], 1, trainX.shape[1]))
testX = np.reshape(testX, (testX.shape[0], 1, testX.shape[1]))

# create and fit the LSTM network
model = Sequential()
model.add(LSTM(4, input_shape=(1, look_back)))
model.add(Dense(1))
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(trainX, trainY, epochs=EPOCHS, batch_size=1, verbose=2)

# make predictions
trainPredict = model.predict(trainX)
testPredict = model.predict(testX)

# invert predictions
trainPredict = scaler.inverse_transform(trainPredict)
trainY = scaler.inverse_transform([trainY])
testPredict = scaler.inverse_transform(testPredict)
testY = scaler.inverse_transform([testY])

# calculate root mean squared error
trainScore = np.sqrt(mean_squared_error(trainY[0], trainPredict[:,0]))
print('Train Score: %.2f RMSE' % (trainScore))
testScore = np.sqrt(mean_squared_error(testY[0], testPredict[:,0]))
print('Test Score: %.2f RMSE' % (testScore))

# shift train predictions for plotting
trainPredictPlot = np.empty_like(dataset)
trainPredictPlot[:, :] = np.nan
trainPredictPlot[look_back:len(trainPredict)+look_back, :] = trainPredict

# shift test predictions for plotting
testPredictPlot = np.empty_like(dataset)
testPredictPlot[:, :] = np.nan
testPredictPlot[len(trainPredict)+(look_back*2)+1:len(dataset)-1, :] = testPredict

# plot baseline and predictions
plt.plot(scaler.inverse_transform(dataset))
plt.plot(trainPredictPlot)
plt.plot(testPredictPlot)
plt.show()

Where it seems to work:

But how do I predict or forecast into the future an entire day? I have another days worth of data that the model hasnt seen before:

testday = read_csv('https://raw.githubusercontent.com/bbartling/Data/master/test_day.csv', index_col=[0], parse_dates=True)
testday_scaled = scaler.fit_transform(testday.values)

testday_scaled = np.swapaxes(testday_scaled, 0, 1)

testday_scaled = testday_scaled[None, ...]
testday_predict = model.predict(testday_scaled)
testday_predict = scaler.inverse_transform(testday_predict)

print(testday_predict[0][0])

For example the print is only 1 step into the future. 39.37902 how do I predict out 24 hours or a whole day ahead into the future?

Answer 1

You can use the prediction of the network as an actual value, and create a new input tensor using the previous input with a new column to the right (and removing the first column, since you imposed the network input to be 96-column wide). Then, you can use such a tensor as input to the model, getting another value. If you repeat this process multiple times (autoregressively), you can predict any further you may want.

Answer 2

So this is what I have to predict 96 samples into the future which is 24 hours 15 minutes at a crack:

testday = read_csv('https://raw.githubusercontent.com/bbartling/Data/master/test_day.csv', index_col=[0], parse_dates=True)
testday_copy = testday.copy()

testday_scaled = scaler.fit_transform(testday.values)
testday_scaled = np.swapaxes(testday_scaled, 0, 1)

testday_scaled = testday_scaled[None, ...]
testday_predict = model.predict(testday_scaled)
testday_predict = scaler.inverse_transform(testday_predict)

print("test prediction is: ",testday_predict)


for i in range(96):

    last_index_stamp = testday.last_valid_index()
    print("last_index_stamp: ",last_index_stamp, " i: ",i)
    
    new_timestamp = last_index_stamp + timedelta(minutes=15)
    print("new_timestamp: ",new_timestamp, " i: ",i)
    
    testday_scaled = scaler.fit_transform(testday.values)
    testday_scaled = np.swapaxes(testday_scaled, 0, 1)
    
    testday_scaled = testday_scaled[None, ...]
    testday_predict = model.predict(testday_scaled)
    testday_predict = scaler.inverse_transform(testday_predict)

    print("prediction is: ",testday_predict[0][0])
    
    # add new row of timestamp and prediction to end of the df
    testday.loc[new_timestamp,:] = testday_predict[0][0]
    
    # remove first df row
    testday = testday.iloc[1:]
    
    print("*******************************************")

The basics of the code above (maintain the model input shape) is for each time step remove the first row of the pandas df and then adds the prediction to the end in the for loop...one prediction at a time to complete an entire days. Where at first glance seems to work nice:

testday.plot(figsize=(15, 7), title='Future 24 hours electricity')

Like I am happy with the results: