Why MLP only learns bias for unbalanced binary classification?

Question

I just started training a MLP model on a data set which has the following statistics. Notice that both train and validation sets are unbalanced (88.4% negatives).

Note: The unbalance isn't caused by sampling and is because that's the natural ratio of the classes. E.g. when I look back in a 3 month period, I see I have eaten 88 apples and 12 oranges; and looking forward, I might still favor apples more than oranges by the same ratio. I may be wrong here but I think I shouldn't give class weights nor resample to balance the two classes, because that's their natural ratio.

Train data:
    Shape (features): (891473, 122)
    Shape (labels): (891473,)
    # of classes: [(0, 788118), (1, 103355)]
    % of classes: [(0, 0.88406266931247501), (1, 0.11593733068752503)]
Validation data:
    Shape (features): (251141, 122)
    Shape (labels): (251141,)
    # of classes: [(0, 222009), (1, 29132)]
    % of classes: [(0, 0.8840014175303913), (1, 0.1159985824696087)]

I'm using three 8-sized dense layers each following a ReLU layer, finally a sigmoid since it's binary classification.

However, the training process converged after only 2 epochs and basically only learned the bias term and and ended up predicting everything to be negative. The confusion matrix stays unchanged like this since the 2nd epoch till the end (20 epochs):

Train confusion matrix:
           P = 0   P = 1 | Total
    L = 0  0.884   0.000 | 0.884
    L = 1  0.116   0.000 | 0.116
    ----------------------------
    Total  1.000   0.000 | 0.884
Validation confusion matrix:
           P = 0   P = 1 | Total
    L = 0  0.884   0.000 | 0.884
    L = 1  0.116   0.000 | 0.116
    ----------------------------
    Total  1.000   0.000 | 0.884

I have tried to increase the size of the dense layer (e.g. from 32, 128, 512, 1024). They all yielded the same result.

I'd expect that with 3 x 1024 neurons the model would be complex enough to learn something beyond a single bias term. So I'm quite confused as what I am doing wrong to make it not learning anything other than the bias.

Question:

1. What could be the possible reasons for the model to learn not much?
2. What should I try and/or what kind of info should I be looking for at this point to move this forward?

Thank you for your time in advance~!

The following is a snippet of my code to show what I was trying to do:

learning_rate = 1e-4
batch_size = 10000
epochs = 5

DENSE_SIZE = 1024

train = (training_features.values, training_labels.values)
validation = (holdout_features.values, holdout_labels.values)
test = (eval_features.values, eval_labels.values)
eval_sets = [
    ('Train', train),
    ('Validation', validation),
    ('Test', test),
]

model = Sequential()
model.add(Dense(DENSE_SIZE, activation='relu', input_shape=train[0].shape[1:]))
model.add(Dense(DENSE_SIZE, activation='relu'))
model.add(Dense(DENSE_SIZE, activation='relu'))
model.add(Dense(1, activation='sigmoid'))

# initiate optimizer
opt = RMSprop(lr=learning_rate)

# Let's train the model
model.compile(loss='binary_crossentropy', optimizer=opt, metrics=['accuracy'])

model.fit(
    train[0],
    train[1],
    batch_size=batch_size,
    epochs=epochs,
    validation_data=validation,
    shuffle=True,
    verbose=2,
    callbacks=callbacks,
)

EDIT:

The three answers as of writing all suggest that the inefficiency in learning is associated w/ the imbalanced classes and I should try to balance the classes by either using class weights or resampling.

While I'm happy to see an apparent agreement of what the problem is what , could anyone provide more insights about why imbalanced classes cause difficulty even for a complex model? In theory a complex enough model can at least memorize all the rarely positive samples and achieve a higher-than-bias accuracy on the training set.

Answer 1

Generally there are a number of ways to deal with class imbalance (more details here fore example).

Specifically in your case, I would either try training with the training samples evenly distributed in classes or I would play around changing the loss function, to take into account the different classes' support. For the latter, you can check this discussion also .

Answer 2

Though you know that dataset is naturally balanced but your model will still consider it as unbalanced dataset. If you take a example financial fraud detection, number of fraudulent transactions are always less than the ligit translations, to handle such such cases data scientists will have to balance the classes for better results. You will have to balance your dataset to get the better results from model.

Answer 3

It is not a matter of model complexity. In order to address class imbalance, you should specify class weights as argument to the fit function in order for the logits of the cross entropy to be scaled accordingly.

You can find further info in stackoverflow and examples on how to provide the weights to fit here.