0
$\begingroup$

So, I'm trying to build a LSTM model to classify multiclass text label. The goal is to make a prediction about user rating (1, 2, 3, 4, 5) based on their review.

My hyperparameter is like this:

# set hyperparameter
vocab_size = 5000 # make the top list of words (common words)
embedding_dim = 32
max_length = 354
oov_tok = '<OOV>' # OOV = Out of Vocabulary

Model:

model = Sequential()
model.add(Embedding(vocab_size, embedding_dim))
model.add(Dropout(0.5))
model.add(Bidirectional(LSTM(embedding_dim)))
model.add(Dense(1, activation='softmax'))
model.summary()

Compile:

model.compile(
    loss='categorical_crossentropy',
    optimizer='Adam',
    metrics=['accuracy']
)

Then, the result I get:

my_lstm_result

I only get 50% accuracy and I think it's not a good result. Any idea how can I improve the accuracy? Here's the collab: Collab's

$\endgroup$
1
$\begingroup$

50% is quite decent because you have five labels and random guessing model would have achieved only 20% accuracy. So you know your model is learning something.

The other thing you want to check out is whether this is suited to be a regression problem more than classification. For e.g, misclassifying a 5 (ground truth) into a 4 is better than misclassifying into a 2. Your current loss function treats both situations as equally bad. So, it's possible that your current accuracy measure is underestimating the models true performance.

You should also try out simple naive bayes model on top of bag of words. It will be interesting to compare how that compares with the more complex lstm.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.