I am following the Google tutorial on ML for text classification

I made this Google Colab notebook which you should be able to run from start to finish to see the issue.

  • When the code trains a sequential CNN on IMDB data, the loss doesn't decrease and training stops (due to EarlyStopping).
  • When the same code trains a sequential CNN on Rotten Tomato data, the loss decreases (as expected)

At first I could not understand why the IMDB loss failed to decrease, was it the data, the model, something else?

Then I tried Rotten Tomato data and found loss did decrease.

This suggests the data is the problem. I will try to answer my own question by checking the difference in IMDB data (load_imdb_sentiment_analysis_dataset(...)) vs Rotten Tomato data (load_rotten_tomatoes_sentiment_analysis_dataset(...)).

a screenshot of this Google Colab notebook, showing a CNN where loss does not decrease for IMDB data, but a CNN where loss does decrease for Rotten Tomatoes data https://gist.github.com/theredpea/52d7ab108339636a467f2feb063338bc


I notice the Rotten Tomatoes data labels have 5 distinct values, creating a multiclass problem

 >> array([0, 1, 2, 3, 4])

Whereas the IMDB dataset have only 2 labels; a binary classification problem

 >> array([0, 1])

When I re-train the model on the Rotten Tomatoes dataset, limited to just two labels (class=0, or class==4), then I get the same results (where the loss doesn't decrease from epoch to epoch)... So it's not specific to the data, it's specific to the type of classification problem...

train_li = np.isin(rt_data[0][1], (0,4))
test_li = np.isin(rt_data[1][1], (0,4))
rt_result = None
with tf.device('/device:GPU:0'):
  rt_result = train_sequence_model(  
        (np.array(rt_data[0][0])[train_li], np.where(rt_data[0][1][train_li]==0, 0,1)),
        (np.array(rt_data[1][0])[test_li], np.where(rt_data[1][1][test_li]==0, 0, 1))
    ), epochs=10)
     # ... outputs where the losses don't decrease ...

In general, ML is different in binary classification (num_classes=2) vs multiclass classification problems (num_classes>2). Specifically this notebook behaves differently in at least 3 ways:

First, in choosing the loss function:

if num_classes == 2:
    loss = 'binary_crossentropy'
    loss = 'sparse_categorical_crossentropy'

Second, in choosing the activation function and third, choosing the number of output units for the neural network:

if num_classes == 2:
    activation = 'sigmoid'
    units = 1
    activation = 'softmax'
    units = num_classes

So I think these have something to do with my answer... more later...



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.