I want to make a predictor using Keras LSTM model. I have a sequence of places visited. The task is to predict the last destination.

I went through different examples but it seems I am not able to shape the input properly.

I am stuck on how to prepare in my program my data to give them to the LSTM model. Here is a minimum code related to my problem.

input_csv ='input.csv'
max_features = 6
df = pd.read_csv(input_csv)

#Cafe =0.1, Park =0.2, Shop =0.3, Home=0.4, Movie = 0.5, School=0.6

enter image description here

For example, Person A visited cafe (0.1) -> park(0.2) -> cafe (0.1) -> park(0.2) and finally ended at school(0.6). The desired task is to predict y (end) based on input X (place1,place2,place3,place4).

X = df.iloc[:,1:5].to_numpy() 
X_train = X.reshape(6, 1, 4) # X.reshape(samples, timesteps, features)
#(6, 1, 4)

y = df.iloc[:,-1].to_numpy()
#Building model
model = Sequential()
model.add(LSTM(6, dropout=0.2, recurrent_dropout=0.2, input_shape=(None, 1)))
model.add(Dense(max_features, activation='sigmoid'))

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Train
pred = model.predict(X_train)
predict_classes = np.argmax(pred,axis=1)

ValueError: Error when checking input: expected lstm_4_input to have shape (None, 1) but got array with shape (1, 4)

I would appreciate if someone could help me in clearing my confusion or pointing to some explanations. Thank You.

  • $\begingroup$ Hi, and welcome to datascience.stackexchange.com. Your question is really more about programming, and many similar question on this exact issue appear in Stack Overflow. I suggest searching there (not asking a new question, searching). Good luck! $\endgroup$ Jul 15 '20 at 6:29

Few points that i want to mention :

  • First of all, Here i think the input u are preparing where I see doubt. The objective is to predict end location. The input shape u prepare is a doubt for me because your sequence_length should be "4" and you have an initial hidden dimension if "1". If I am not wrong then your final share should be (batch_size,sequence_length,hiiden_dimension) = (6,4,1). Because this is what dimension shape LSTM is expecting always.
  • Second is change your loss function because u want to do predict from more than one class as mentioned like Cafe =0.1, Park =0.2, Shop =0.3, Home=0.4, Movie = 0.5, School=0.6. Chnage your loss function to categorical_crossentropy in fit() function. Also change activation='sigmoid' to activation='softmax'.

I think these changes u will have to do according to your use case. Please feedback my answer.

  • $\begingroup$ Thank you Gaurav. After applying your suggestions, I got the following error: ValueError: Error when checking target: expected dense_2 to have shape (6,) but got array with shape (1,) $\endgroup$
    – bdur
    Jul 11 '20 at 11:59
  • $\begingroup$ what u will have to do is that get output at all time time stamp in LSTM. There are functionality available in tensorflow for get output at all time stemp.Each timestamp output u will have loop over it and apply individual dense layer on it. U will have to do manually loop in this case. Here the problem i see is that it only take output of last time stamp that's why it return(1,) shape while it expect (6,) according to dense that's it throw an error. $\endgroup$ Jul 13 '20 at 4:13
  • $\begingroup$ Thank you for Gaurav for your guidance. I have posted my initial running code below ( though improvements may be necessary) $\endgroup$
    – bdur
    Jul 14 '20 at 14:02

A minimum running code by modifying the model building section in the previous code:

print('Build model...')
model = Sequential()
#model.add(LSTM(12, input_shape=( X_train.shape[1:])))

model.add(LSTM(6, dropout=0.2, recurrent_dropout=0.2, input_shape=(None, 1)))
model.add(Dense(max_features, activation='softmax'))

# try using different optimizers and different optimizer configs
model.compile(loss='sparse_categorical_crossentropy', #loss='categorical_crossentropy',
  • 1
    $\begingroup$ Thanks for sharing your code here. I think u might have to add embedding layer just above LSTM in order to computer representation of input. I will share my notebook soon. U may take reference of it. It is made for another purpose but I might help. Please look at possibility of add embedding layer on just above LSTM. $\endgroup$ Jul 16 '20 at 8:23
  • $\begingroup$ Thank you for your notebook in advance. $\endgroup$
    – bdur
    Jul 19 '20 at 10:59
  • 1
    $\begingroup$ github.com/gauravkoradiya/Tensorflow-Specialization/tree/master/…. Kindly look at his notebook and see how LSTM or RNN is used. Just see model IO. $\endgroup$ Jul 20 '20 at 9:45
  • 1
    $\begingroup$ Its good practice to use the Embdedding layer above LSTM to get out mismatch error. $\endgroup$ Jul 20 '20 at 11:27

Not the answer you're looking for? Browse other questions tagged or ask your own question.