2
$\begingroup$

I used AirPassenger Dataset. And based on several previous values(for examples 20) I want to predict several(3 or 5) steps in future.

Like

X -> y

[10,20,30,....200]->[210,220,230]

[20,30,40,....210]->[220,230,240]

[30,40,50,....220]->[230,240,250] and etc

I made normalization and split.

scaler = StandardScaler()
df['#Passengers'] = scaler.fit_transform(df[['#Passengers']])
train_size = int(len(df)*0.8)
test_size = len(df) -train_size
train,test = df[['#Passengers']].iloc[:train_size], df[['#Passengers']].iloc[train_size:]

create dataset

def create_dataset(X, y, time_steps=1,pred_range=1):
    Xs, ys = [], []
    for i in range(len(X) - time_steps-pred_range):
        v = X.iloc[i:(i + time_steps)].values
        Xs.append(v)
        ys.append(y.iloc[i + time_steps:i + time_steps+pred_range])
    return np.array(Xs), np.array(ys)

time_steps = 20
pred_range = 5
xtrain, ytrain = create_dataset(train, train, time_steps,pred_range)
xtest, ytest = create_dataset(test, test, time_steps,pred_range)
input_shape=(xtrain.shape[1], xtrain.shape[2])

and made on Keras simple model.

mcp_save = tf.keras.callbacks.ModelCheckpoint('.mdl_wts.hdf5', save_best_only=True, monitor='val_loss', mode='min')
reduce_lr_loss = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=7, verbose=1, epsilon=1e-4, mode='min')
es_callback = keras.callbacks.EarlyStopping(monitor='val_loss', patience=10)
output_size=pred_range
model = keras.Sequential()
model.add(keras.layers.LSTM(256, return_sequences=True, input_shape=input_shape,))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.LSTM(64, return_sequences=False))
model.add(keras.layers.Dropout(0.1))
model.add(keras.layers.Dense(output_size))
model.compile(loss='mae', optimizer='adam',metrics =["mean_squared_error","mae"])
history = model.fit(xtrain, ytrain, epochs=50, batch_size=100, validation_data=(xtest, ytest), verbose=2, shuffle=False,callbacks=[mcp_save,reduce_lr_loss,es_callback])

Problem: When I made prediction using any points from test data, I got a lines ,with mostly same shape

yhat3 = model.predict(np.array([xtest[3]]))
yhat30 = model.predict(np.array([xtest[30]]))

Predition yhat3

Prediction yhat30

As you see, red line(prediction) by shape is the same, no matter what is xtest

And when you plot this to whole test data of Airpassenger it look like Whole

Additional question why LSTM not catching Amplitutes of curve? Is this because of my bad Model, or because of normalization, or because of LSTM , or I need to separate trend and seasonality? enter image description here

$\endgroup$
5
  • $\begingroup$ You can't make a prediction with just one point: LSTMs are built to make predictions based on a serie of values in order to detect their dynamic and be able to make reliable predictions. Consequently, you should take a serie of values (at least 30 in your case) to make predictions. $\endgroup$ Aug 27, 2021 at 13:42
  • $\begingroup$ Based on code, i used several points to predict ,see time_steps = 20 , v = X.iloc[i:(i + time_steps)].values $\endgroup$ Aug 27, 2021 at 14:28
  • $\begingroup$ Have you tried more time steps? 20 might be not enough to make reliable predictions. On the other hand, have you tried to use relative values (i.e. increased by 1.5, then decrease by 2, etc.) instead of absolute ones? In this way your NN wil only focus on data dynamics. $\endgroup$ Aug 30, 2021 at 13:14
  • $\begingroup$ I found a solution. I worked more deeply with Scaling and feature transformation. and it's helps $\endgroup$ Aug 30, 2021 at 21:30
  • $\begingroup$ That's great. I recommend you to study the LSTM publication in order to understand how it works, it's very interesting. For instance, there is a feeding loop called "constant error carroussel" that keep the learning process on data's dynamics, which could explain why it couldn't learn so well on absolute values. Note: If a comment has been usefull, don't hesitate to upvote it. researchgate.net/publication/13853244_Long_Short-term_Memory $\endgroup$ Sep 1, 2021 at 7:41

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.