# Value Error: Operands could not be broadcast together with shapes - LSTM

I am trying an LSTM model using tensorflow following this tutorial .

I am having trouble understanding why am I getting an error in my test set when I try to invert scaling for forecast (line 86 in the tutorial).

After loading the dataset and created featured, following is what I did for reframed dataset,

# split into train and test sets
values = reframed.values
split_point = len(reframed)- 168

train = values[ : split_point, :]
test = values[split_point: , :]

# split into input and outputs
train_X, train_y = train[:, :-1], train[:, -1]
test_X, test_y = test[:, :-1], test[:, -1]

# reshape input to be 3D [samples, timesteps, features]
train_X = train_X.reshape((train_X.shape[0], 1, train_X.shape[1]))
test_X = test_X.reshape((test_X.shape[0], 1, test_X.shape[1]))

print(train_X.shape, train_y.shape, test_X.shape, test_y.shape)
>> (2399, 1, 39) (2399,) (168, 1, 39) (168,)

# design network
model = Sequential()

# fit network
history = model.fit(train_X, train_y, epochs=50, batch_size=72, validation_data=(test_X, test_y), verbose=2, shuffle=False)

# make a prediction
yhat = model.predict(test_X)
test_X = test_X.reshape((test_X.shape[0], test_X.shape[2]))

inv_yhat = np.concatenate((yhat, test_X[:, 1:]), axis=1)


After this step, the following code that has scaler.invert_transform throws the error.

inv_yhat = scaler.inverse_transform(inv_yhat)


ValueError: operands could not be broadcast together with shapes (168,39) (41,) (168,39)

Shape,

test_X.shape
>> (168, 1, 39)

yhat.shape
>>(168,1)

inv_yhat.shape
>> (168,39)


Any help or sugeestions would be appreciated!

• Better to include the code sample (or at least part of it) here. External links disappear over time. – Steve Kallestad Aug 22 '17 at 20:02
• @SteveKallestad thank you for the response. sure, I can include it. It will be really long about 90 lines of code. – i.n.n.m Aug 22 '17 at 20:05
• The error occurs because of the variant in the shapes. I don't have time to dig into it now, but I would have a look at the data and see if it hasn't changed or wasn't prepared correctly. – Steve Kallestad Aug 22 '17 at 20:07
• @SteveKallestad i agree with you, if you have time, there's a question about this on the tutorial and the author asked that person to check the data preparation. I went back and checked mine, print(train_X.shape, train_y.shape, test_X.shape, test_y.shape) and I got (2399, 1, 39) (2399,) (168, 1, 39) (168,) like this. – i.n.n.m Aug 22 '17 at 20:11
• @SteveKallestad thank you for your suggestion, I could find the error and it works perfect now. I posted my mistake as an answer, in case if someone comes across this issue! – i.n.n.m Aug 23 '17 at 21:54

The answer that OP provided is correct, yet I would like to elaborate a little more on it, in an attempt to shed more light.

First of all, you have to understand what is performed by the call

reframed = series_to_supervised(scaled, 1, 1)


in line 50.

Shortly, let us assume that the input has n samples of m features (variables). This means that the shape of the scaled dataframe is (n) x (m). The particular call to series_to_supervised() in line 50 implies that the scaled dataframe will be reshaped into a (n-1) x (2m) dataframe, to include the current and the 1-step look-back values, after dropping the row of NaN values due to the shift operation inside the body of series_to_supervised(). Most likely, your error is caused by line 52:

reframed.drop(reframed.columns[[9,10,11,12,13,14,15]], axis=1, inplace=True)


because you are probably dropping an arbitrary number of columns. The code is structured so that it forecasts one time series based on the past values of multiple time series (including the signal of interest). Therefore, you have to drop (m-1) columns, so that in the end all m past values persist, along with the current value of the signal of interest that will be used as ground truth. This means that the shape of reframed should be (n-1) x (m+1) after line 52. If not, you probably get that error with the dimension mismatch when reverting the scaling.

At this point I would like to emphasize on the fact that you should be very careful with logical errors when dropping columns, because you want to drop all but the signal of interest in line 52. If you keep any other column the code will still run, but you will certainly get logical artifacts.

Moreover, if you drop any column other than the first one (at index 0), you have to modify the concatenation in lines 85 & 90 accordingly, otherwise you will also get logical artifacts.

I thought of posting as an answer according to the suggestion by @Seteve Kallestad in case of someone come across this issue.

The answer is: The shape of the data must be the same when inverting the scale as when it was originally scaled.

I was making an error in my data preparation. Initially, when I scaled the data my test set had a shape of (168,39).

As it mentions in the error

shapes (168,39) (41,) (168,39)

I had 41 columns and it should have been 39.

The reason is shape of yhat is (168,1) and shape of test_X[:, 1:] is 168,38 as it starts from column number 1. When they are concatenated it becomes (168,39) which is exactly how it was when it was scaled.

• I am still struggling with the same error and I can't seem to understand the answer you posted. Can you please let me know how you managed to bypass this error?? – Effa Jun 12 '18 at 5:18
• Hi Effa. Are you using the exact same tutorial code and you just changed the dataset? – pcko1 Jun 12 '18 at 7:48
• @Effa Trick is to always check dimensionality of your data frames (training and test sets). #pcko1 answer explains well. – i.n.n.m Jun 12 '18 at 14:13
• If you have more or less features than in the exampe, then in the second version at the bottom of the tutorial, replace '-7' with '(n_features-1)' in lines 88 and 93 i.e. inv_yhat = concatenate((yhat, test_X[:, -(n_features-1):]), axis=1). This corrects the dimensions. – intotecho Feb 5 '19 at 3:40