How to Work with Imbalanced Data

Question

I am building a binary classifier from a set of feature vectors some of which are categorical like Yes or No (two options). I am replacing them with 1 and 0 and since there is strong imbalance between 1 and 0 my model learns basically on those features. I am afraid that this particular feature might be an artifact - samples that have 0 can also sometimes be classified as 1. What to do in this situation? Should I drop the feature completely?

Update:

Let me explain in detail the problem since the answers mostly focus on imbalanced classes. I have a dataset that comprises of around 30 features and binary classes {0,1}. Features are mostly numerical (continuous), but there are also binary categorical features like YES/NO, MALE/FEMALE etc.

One aspect of this dataset is imbalanced classes (there are more ones than zeros) and the other aspect is that one categorical feature, lets say $x$ (YES/NO) is not very balanced also. In fact if you make predictions solely based on $x$ like: $x = 1 \rightarrow 1, x=0 \rightarrow 0$ you will perform better than a naive model, which predicts only $0$ (remember imbalanced classes).

Now, my dilemma is what to do in that case? Should I remove that variable completely from modelling or maybe use some techniques of bias removal?

Answer 1

Generally, there are three main solutions for imbalance data classification.

1. Over-sampling the thin class by generating artificial data or some other ways. This solution can be helpful in situations in which generating accurate artificial data is possible.
2. Under-sampling the populous class. This solution also can be suitable for situations in which the ratio of classes population is not very low (for example 30% to 70% rather than 1% to 99%)
3. Using a proper loss function which can handle imbalanced data. There are many metrics that can evaluate the performance of the classifiers at the presence of imbalanced data.

Disclaimer: If you use python, PyCM module can help you to find out these metrics.

Here is a simple code to get the recommended parameters from this module:

>>> from pycm import *

>>> cm = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2":2}, "Class2": {"Class1": 0, "Class2": 5}})  

>>> print(cm.recommended_list)
["Kappa", "SOA1(Landis & Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN", "MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC", "AUCI", "G", "DP", "DPI", "GI"]

After that, each of these parameters you want to use as the loss function can be used as follows:

>>> y_pred = model.predict      #the prediction of the implemented model

>>> y_actu = data.target        #data labels

>>> cm = ConfusionMatrix(y_actu, y_pred)

>>> loss = cm.Kappa             #or any other parameter (Example: cm.SOA1)

Answer 2

In some cases, the classification problems will have imbalanced data (Either 1 or 0 will be present dominantly in the target column). If we proceed with this data, our created model will have a bias towards the dominant output. To address this problem, we have so many over-sampling techniques to balance the data. Some of them are:

1. Using the resample technique of pandas in python.
2. SMOTE (Package available in python )
3. SMOTE+TOMEK
4. SMOTE+ENN
5. Random Over Sampling

(For 3,4,5 in python, we need to install the imblearn package in python)