How to improve LSTM accuracy on multiclass text classification?

Question

So, I'm trying to build a LSTM model to classify multiclass text label. The goal is to make a prediction about user rating (1, 2, 3, 4, 5) based on their review.

My hyperparameter is like this:

# set hyperparameter
vocab_size = 5000 # make the top list of words (common words)
embedding_dim = 32
max_length = 354
oov_tok = '<OOV>' # OOV = Out of Vocabulary

Model:

model = Sequential()
model.add(Embedding(vocab_size, embedding_dim))
model.add(Dropout(0.5))
model.add(Bidirectional(LSTM(embedding_dim)))
model.add(Dense(1, activation='softmax'))
model.summary()

Compile:

model.compile(
    loss='categorical_crossentropy',
    optimizer='Adam',
    metrics=['accuracy']
)

Then, the result I get:

I only get 50% accuracy and I think it's not a good result. Any idea how can I improve the accuracy? Here's the collab: Collab's

Answer 1

50% is quite decent because you have five labels and random guessing model would have achieved only 20% accuracy. So you know your model is learning something.

The other thing you want to check out is whether this is suited to be a regression problem more than classification. For e.g, misclassifying a 5 (ground truth) into a 4 is better than misclassifying into a 2. Your current loss function treats both situations as equally bad. So, it's possible that your current accuracy measure is underestimating the models true performance.

You should also try out simple naive bayes model on top of bag of words. It will be interesting to compare how that compares with the more complex lstm.