1
$\begingroup$

I am trying to use LSTMs to train and predict authors using reviews data and metadata

author  phone  country day     review 
james   iphone chile   tuesday the book was really amazing

How do I pass all these features into the network?

$\endgroup$
0
$\begingroup$

Since the review of an author is bound to change in terms of the number of words being used in the review, I would suggest using a Keras Sequential() model to build an LSTM encoder for the review itself. The final hidden layer of the review LSTM encoder can then be fed into another LSTM encoder with 3 words (phone, country, and day). Think of the last LSTM encoder as a sequential 3 worded message. The final layer of this LSTM can then be joined with a softmax layer to predict the author.

The reason I suggested two different LSTMs has something to do with word embeddings. Word embeddings are basically features that get modified in multiple hidden layers in an LSTM. Having two different word embeddings means that the same word in the Review Embedding may mean different in the other LSTM embedding, ie have different embedding vectors after training. It ensures that the features are independent. You can argue that you might need 4 different LSTM and Embedding Layers because of 4 different features. That is definitely a viable option, but it might need a lot more time to train. Having more LSTMs will significantly add hyperparameters and the training time will be really high.

That being said, you may not need an LSTM to solve this problem. You can easily convert the phone, country, and day values into integers. There are 7 days in a week so the value of day can only be one from 0 to 6. That reduces the dimension of features significantly thereby reducing the training time. You can probably try a TF-IDF for the review feature or a naive bag of word model and use a Random Forest or SVM Classifier for the prediction of Author. It would be interesting to perform multiple experiments with and without LSTMs and observe results with accuracy and performance.

$\endgroup$
1
  • $\begingroup$ I agree. Just use one to Encode the review. That makes sense. Pad the reviews and send them into a masked enocding layer and then LSTM. Grab the state of that and concatenate with the other measures. Phone, country, day are not sequential so it doesn't make sense to use a LSTM really. As you say, just convert them to normalized features. You could then just place a FCN on the end and reduce down to Softmax. $\endgroup$ Mar 16 '21 at 18:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.