I used AirPassenger Dataset. And based on several previous values(for examples 20) I want to predict several(3 or 5) steps in future.
Like
X -> y
[10,20,30,....200]->[210,220,230]
[20,30,40,....210]->[220,230,240]
[30,40,50,....220]->[230,240,250] and etc
I made normalization and split.
scaler = StandardScaler()
df['#Passengers'] = scaler.fit_transform(df[['#Passengers']])
train_size = int(len(df)*0.8)
test_size = len(df) -train_size
train,test = df[['#Passengers']].iloc[:train_size], df[['#Passengers']].iloc[train_size:]
create dataset
def create_dataset(X, y, time_steps=1,pred_range=1):
Xs, ys = [], []
for i in range(len(X) - time_steps-pred_range):
v = X.iloc[i:(i + time_steps)].values
Xs.append(v)
ys.append(y.iloc[i + time_steps:i + time_steps+pred_range])
return np.array(Xs), np.array(ys)
time_steps = 20
pred_range = 5
xtrain, ytrain = create_dataset(train, train, time_steps,pred_range)
xtest, ytest = create_dataset(test, test, time_steps,pred_range)
input_shape=(xtrain.shape[1], xtrain.shape[2])
and made on Keras simple model.
mcp_save = tf.keras.callbacks.ModelCheckpoint('.mdl_wts.hdf5', save_best_only=True, monitor='val_loss', mode='min')
reduce_lr_loss = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=7, verbose=1, epsilon=1e-4, mode='min')
es_callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
output_size=pred_range
model = keras.Sequential()
model.add(keras.layers.LSTM(256, return_sequences=True, input_shape=input_shape,))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.LSTM(64, return_sequences=False))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(output_size))
model.compile(loss='mae', optimizer='adam',metrics =["mean_squared_error","mae"])
history = model.fit(xtrain, ytrain, epochs=50, batch_size=100, validation_data=(xtest, ytest), verbose=2, shuffle=False,callbacks=[mcp_save,reduce_lr_loss,es_callback])
Problem: When I made prediction using any points from test data, I got a lines ,with mostly same shape
yhat3 = model.predict(np.array([xtest[3]]))
yhat30 = model.predict(np.array([xtest[30]]))
As you see, red line(prediction) by shape is the same, no matter what is xtest
And when you plot this to whole test data of Airpassenger it look like
Additional question why LSTM not catching Amplitutes of curve?
Is this because of my bad Model, or because of normalization, or because of LSTM , or I need to separate trend and seasonality?
