Dealing with biased binary classifier

Question

My training data is weighed heavier on the '1' class, with about a 4:6 ratio. This outputs a classifier that is of 82% accuracy with an emphasis on the '1' class, which makes sense.

Confusion Matrix - 
[[333 133]
 [ 62 612]]

I have the test proportions as well, in which the data will be tested on, which is 0.3 of '1' and 0.7 of '0' or 1900 0s and 900 1s. My classifier outputs 1400 1s and 1300 0s.

My theory is that I need to build a classifier that favours the '0', If so how can I make the classifier biased to one class over another?

I have tried to used the class weights, this does increase the '0' predictions but only by a very small percentage.

Answer 1

For binary classification you can easily adjust the decision threshold to bias the classifier on either class. By default the decision threshold is 0.5. You can easily change that to e.g. 0.55 to get more 0s predictions (i.e.. if p<0.55 then 0, else 1). Such decision threshold adjustment, also called threshold moving or bias/gain adaptation, leads to ROC curve and AUC theory. Have a look here for more info: Threshold moving

Answer 2

What you have in your data called imbalanced classes

From Datacamp

Imbalanced data typically refers to classification tasks where the classes are not represented equally.

For example, you may have a binary classification problem with 100 instances out of which 80 instances are labeled with Class-1, and the remaining 20 instances are marked with Class-2.

In this link, you can find a nice article that explains more what it is and how you can handle it -> https://www.datacamp.com/community/tutorials/diving-deep-imbalanced-data

One of the solutions is to use over or under-sampling. You can achieve this with the SMOTE algorithm. Here is an example in Python.

from imblearn.over_sampling import SMOTE
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=0)

sm = SMOTE(random_state=2)
X_train_res, y_train_res = sm.fit_sample(X_train, y_train.ravel())