Model does not predict anything as positive

Question

I was trying to train a model to predict if someone exists in a speech or not, first I extracted some set of features using a previous model, now I have a 1D vector of features for each example and I want to predict if someone exists, also the data is multilabel, so there are examples with more than one person (more the one '1' in the label vector). But the following happened:

It continually predict things as negative, I tried to change the model (making it deeper, shallower, change learning rate, change the activation functions) and nothing change.

So what's going on? and how to solve this issue?

I use BinaryCrossentropy loss with with_logits=True. And here is a summary of my model:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d (Conv1D)              (None, 3194, 256)         2048      
_________________________________________________________________
leaky_re_lu (LeakyReLU)      (None, 3194, 256)         0         
_________________________________________________________________
batch_normalization (BatchNo (None, 3194, 256)         1024      
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 3190, 256)         327936    
_________________________________________________________________
leaky_re_lu_1 (LeakyReLU)    (None, 3190, 256)         0         
_________________________________________________________________
batch_normalization_1 (Batch (None, 3190, 256)         1024      
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 638, 256)          0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, 634, 128)          163968    
_________________________________________________________________
leaky_re_lu_2 (LeakyReLU)    (None, 634, 128)          0         
_________________________________________________________________
batch_normalization_2 (Batch (None, 634, 128)          512       
_________________________________________________________________
conv1d_3 (Conv1D)            (None, 632, 64)           24640     
_________________________________________________________________
leaky_re_lu_3 (LeakyReLU)    (None, 632, 64)           0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 632, 64)           256       
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 158, 64)           0         
_________________________________________________________________
flatten (Flatten)            (None, 10112)             0         
_________________________________________________________________
dense (Dense)                (None, 64)                647232    
_________________________________________________________________
dense_1 (Dense)              (None, 1251)              81315     
=================================================================
Total params: 1,249,955
Trainable params: 1,248,547
Non-trainable params: 1,408

#Edit: I've tried after the answer from @Khanis to use a weighted cross entropy and things worked, but still there is too much slow learning, is that because of the huge amount of persons? if I decrease them to 100 will the problem solved?

Answer 1

Maybe you should train the model for more epochs

Or use the Weighted Binary Cross Entropy loss with a bigger weight to the Positive label

It would be easier to visualize with an input-output example, but by what you said you have a 1D vector of input features and a 1D vector of binary labels

Up to epoch 3 your model is still predicting positive labels, given by the false_positive rate > 0, it just doesn't get it right, and it is predicting less values as positives each epoch

Given the epoch log we can see that the target class is rare, on the first epoch you show

FN:2539
FP:121536
TP:131
TN:2377794

which gives

P = FN+TP = 2539+131 = 2670
N = FP+TN = 121536+2377794 = 2499330

So the positive class is rougthly 0.1% of the cases

Given that, a model that always predict Negative doesn't seem so bad

And you can see both the loss and the average error are decreasing

However, there will be a point where this can't get any better and the model may start to get things right, you just need to give it more than 10 epochs. It also depends on the quality of the previous model that extracts the features.