Why doesn't loss decrease with each epoch (for IMDB data vs Rotten Tomatoes data)

Question

I am following the Google tutorial on ML for text classification

I made this Google Colab notebook which you should be able to run from start to finish to see the issue.

• When the code trains a sequential CNN on IMDB data, the loss doesn't decrease and training stops (due to EarlyStopping).
• When the same code trains a sequential CNN on Rotten Tomato data, the loss decreases (as expected)

At first I could not understand why the IMDB loss failed to decrease, was it the data, the model, something else?

Then I tried Rotten Tomato data and found loss did decrease.

This suggests the data is the problem. I will try to answer my own question by checking the difference in IMDB data (load_imdb_sentiment_analysis_dataset(...)) vs Rotten Tomato data (load_rotten_tomatoes_sentiment_analysis_dataset(...)).

EDIT

I notice the Rotten Tomatoes data labels have 5 distinct values, creating a multiclass problem

 np.unique(rt_labels)
 >> array([0, 1, 2, 3, 4])

Whereas the IMDB dataset have only 2 labels; a binary classification problem

 np.unique(imdb_labels)
 >> array([0, 1])

When I re-train the model on the Rotten Tomatoes dataset, limited to just two labels (class=0, or class==4), then I get the same results (where the loss doesn't decrease from epoch to epoch)... So it's not specific to the data, it's specific to the type of classification problem...

train_li = np.isin(rt_data[0][1], (0,4))
test_li = np.isin(rt_data[1][1], (0,4))
rt_result = None
with tf.device('/device:GPU:0'):
  rt_result = train_sequence_model(  
    (
        (np.array(rt_data[0][0])[train_li], np.where(rt_data[0][1][train_li]==0, 0,1)),
        (np.array(rt_data[1][0])[test_li], np.where(rt_data[1][1][test_li]==0, 0, 1))
    ), epochs=10)
     # ... outputs where the losses don't decrease ...

In general, ML is different in binary classification (num_classes=2) vs multiclass classification problems (num_classes>2). Specifically this notebook behaves differently in at least 3 ways:

First, in choosing the loss function:

if num_classes == 2:
    loss = 'binary_crossentropy'
else:
    loss = 'sparse_categorical_crossentropy'

Second, in choosing the activation function and third, choosing the number of output units for the neural network:

if num_classes == 2:
    activation = 'sigmoid'
    units = 1
else:
    activation = 'softmax'
    units = num_classes

So I think these have something to do with my answer... more later...