I am following the Google tutorial on ML for text classification
I made this Google Colab notebook which you should be able to run from start to finish to see the issue.
- When the code trains a sequential CNN on IMDB data, the loss doesn't decrease and training stops (due to EarlyStopping).
- When the same code trains a sequential CNN on Rotten Tomato data, the loss decreases (as expected)
At first I could not understand why the IMDB loss failed to decrease, was it the data, the model, something else?
Then I tried Rotten Tomato data and found loss did decrease.
This suggests the data is the problem. I will try to answer my own question by checking the difference in IMDB data (
load_imdb_sentiment_analysis_dataset(...)) vs Rotten Tomato data (
I notice the Rotten Tomatoes data labels have 5 distinct values, creating a multiclass problem
np.unique(rt_labels) >> array([0, 1, 2, 3, 4])
Whereas the IMDB dataset have only 2 labels; a binary classification problem
np.unique(imdb_labels) >> array([0, 1])
When I re-train the model on the Rotten Tomatoes dataset, limited to just two labels (
class==4), then I get the same results (where the loss doesn't decrease from epoch to epoch)... So it's not specific to the data, it's specific to the type of classification problem...
train_li = np.isin(rt_data, (0,4)) test_li = np.isin(rt_data, (0,4)) rt_result = None with tf.device('/device:GPU:0'): rt_result = train_sequence_model( ( (np.array(rt_data)[train_li], np.where(rt_data[train_li]==0, 0,1)), (np.array(rt_data)[test_li], np.where(rt_data[test_li]==0, 0, 1)) ), epochs=10) # ... outputs where the losses don't decrease ...
In general, ML is different in binary classification (
num_classes=2) vs multiclass classification problems (
num_classes>2). Specifically this notebook behaves differently in at least 3 ways:
First, in choosing the loss function:
if num_classes == 2: loss = 'binary_crossentropy' else: loss = 'sparse_categorical_crossentropy'
Second, in choosing the activation function and third, choosing the number of output units for the neural network:
if num_classes == 2: activation = 'sigmoid' units = 1 else: activation = 'softmax' units = num_classes
So I think these have something to do with my answer... more later...