I have a very simple LSTM model

model = Sequential()
model.add(LSTM(64, input_shape=(seq_length, X_train.shape[2]) , return_sequences=True))
model.add(Dense(y_cat_train.shape[2], activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
model.fit(X_train, y_cat_train, epochs=100, batch_size=10, verbose=2)

The input X_train has 2 feature , one is categorical (values 1-4) and the other is numeric (values 1-100). There are 4 classes in y_test that I one-hot-encoded with keras's to_categorical .

  1. Should I encode the categorical input feature as well ? If I do , how can I pass it along with the other feature ? (e.g. now a timestep looks like this for example: [1,44])
  2. Later , I would like to take make a sampling , meaning I need to take the predicted y_hat<t-1> and pass it as x<t> . I will have to pass the second numeric feature (1-100) along with it. How can it be done ?

EDIT : note that I do not want my numeric feature to become categorical since there is importance to the values (meaning 2<10<90 etc)


(1) Yes, it is a common practice to encode the categorical feature by one-hot encoding, for example, encode [1,44] as [1,0,0,0,44], encode [2,44] as [0,1,0,0,44], etc.

(2) Same as (1), just concatenate the one-hot encoded categorical feature and the numerical feature(s).

| improve this answer | |
  • $\begingroup$ So it will be treated as one feature ? The categorical part is actuallyy<t-1> so X<t> = [y<t-1>,feature2] - won't concatenating lose some of the importance of one of the features ? $\endgroup$ – M.F Jan 11 '19 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.