LSTM returns the same results for different inputs

Question

Hey everyone,

I am working on a LSTM network in TensorFlow that predicts the values of the price-index of different product-categories in a month, based on those same values of the 12 months before. Unfortunately I ran into some issues while trying to predict. My dataset (data) essentially looks like this:

	Product 1 Price-index	Product 2 Price-index	...	Product 128 Price-index	Product 129 Price-index
Month 1	99.1	99.5	...	100	100.2
Month 2	100.6	101	...	100.3	100.6
...	...	...	...	...	...
Month 305	150	124	...	135	136
Month 306	151	126	...	137	136.2

I did some preprocessing but did not do any rescaling/standardization as the input/outputvalues are all indexes starting from 100 and are therefore on the same scale:

#Split in training and test set
train_size = int(len(data) * 0.9)
test_size = len(data) - train_size
train, test = data.iloc[0:train_size], data.iloc[train_size:len(data)]
print(len(train), len(test))

#Create dataset
def create_dataset(X, time_steps=1):
Xs, ys = [], []
for i in range(len(X) - time_steps):
    v = X.iloc[i:(i + time_steps)].values
    Xs.append(v)
    ys.append(X.iloc[i + time_steps])
return np.array(Xs), np.array(ys)

time_steps = 12
X_train, y_train = create_dataset(train, time_steps)
X_test, y_test = create_dataset(test, time_steps)
print(X_train.shape, y_train.shape)

Which produces the following output:

(263, 12, 129) (263, 129)

My model looks like this:

tf.keras.backend.clear_session

model = tf.keras.Sequential([
  tf.keras.layers.LSTM(50, input_shape=(X_train.shape[1],X_train.shape[2])),
  tf.keras.layers.Dropout(0.2),
  tf.keras.layers.Dense(129)                      
])             

model.compile(loss='mae', optimizer='adam')

history = model.fit(X_train, y_train, epochs=500, batch_size=270, validation_data=(X_test, y_test), verbose=1, shuffle=False)

It gives no error and trains. However, when it is finished training and I try to use predict to get the predicted values for X_train (to plot them against the actual values) I get the following output:

array([[3.1375432, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233],
   [3.1375432, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233],
   [3.1375432, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233],
   ...,
   [3.1375432, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233],
   [3.1375432, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233],
   [3.1375434, 2.9852755, 3.4141188, ..., 3.3334997, 3.0094063,
    2.5739233]], dtype=float32)

Which essentially gives me the exact same output for the different months of prediction. Furthermore, if I try to predict for one month, for example X_train[1] I get the following error from the network:

ValueError: Input 0 of layer sequential_11 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: (None, 129)

While the shape of X_train = (12,129), the same shape that the network should expect. Is there anyone who can point me in the right direction and show me what I am doing wrong?

Thanks!

Answer 1

it seems like scaling my data helped. I refer to the following thread on GitHub:

https://github.com/keras-team/keras/issues/1727