2
$\begingroup$

I am building a binary classifier from a set of feature vectors some of which are categorical like Yes or No (two options). I am replacing them with 1 and 0 and since there is strong imbalance between 1 and 0 my model learns basically on those features. I am afraid that this particular feature might be an artifact - samples that have 0 can also sometimes be classified as 1. What to do in this situation? Should I drop the feature completely?

Update:

Let me explain in detail the problem since the answers mostly focus on imbalanced classes. I have a dataset that comprises of around 30 features and binary classes {0,1}. Features are mostly numerical (continuous), but there are also binary categorical features like YES/NO, MALE/FEMALE etc.

One aspect of this dataset is imbalanced classes (there are more ones than zeros) and the other aspect is that one categorical feature, lets say $x$ (YES/NO) is not very balanced also. In fact if you make predictions solely based on $x$ like: $x = 1 \rightarrow 1, x=0 \rightarrow 0$ you will perform better than a naive model, which predicts only $0$ (remember imbalanced classes).

Now, my dilemma is what to do in that case? Should I remove that variable completely from modelling or maybe use some techniques of bias removal?

| improve this question | | | | |
$\endgroup$
  • $\begingroup$ Use can try using Dropout layers or BatchNormalization layers. $\endgroup$ – Shubham Panchal Mar 5 '19 at 11:25
  • $\begingroup$ Which algo is used in the model ? If this categorical feature is not useful for prediction, algo should learn to ignore the feature. Can you try Tree based algo with some UI tool (Like Weka) and see if this feature is used at all for prediction ? $\endgroup$ – Shamit Verma Mar 6 '19 at 4:15
2
$\begingroup$

Generally, there are three main solutions for imbalance data classification.

  1. Over-sampling the thin class by generating artificial data or some other ways. This solution can be helpful in situations in which generating accurate artificial data is possible.
  2. Under-sampling the populous class. This solution also can be suitable for situations in which the ratio of classes population is not very low (for example 30% to 70% rather than 1% to 99%)
  3. Using a proper loss function which can handle imbalanced data. There are many metrics that can evaluate the performance of the classifiers at the presence of imbalanced data.

Disclaimer: If you use python, PyCM module can help you to find out these metrics.

Here is a simple code to get the recommended parameters from this module:

>>> from pycm import *

>>> cm = ConfusionMatrix(matrix={"Class1": {"Class1": 1, "Class2":2}, "Class2": {"Class1": 0, "Class2": 5}})  

>>> print(cm.recommended_list)
["Kappa", "SOA1(Landis & Koch)", "SOA2(Fleiss)", "SOA3(Altman)", "SOA4(Cicchetti)", "CEN", "MCEN", "MCC", "J", "Overall J", "Overall MCC", "Overall CEN", "Overall MCEN", "AUC", "AUCI", "G", "DP", "DPI", "GI"]

After that, each of these parameters you want to use as the loss function can be used as follows:

>>> y_pred = model.predict      #the prediction of the implemented model

>>> y_actu = data.target        #data labels

>>> cm = ConfusionMatrix(y_actu, y_pred)

>>> loss = cm.Kappa             #or any other parameter (Example: cm.SOA1)
| improve this answer | | | | |
$\endgroup$
0
$\begingroup$

In some cases, the classification problems will have imbalanced data (Either 1 or 0 will be present dominantly in the target column). If we proceed with this data, our created model will have a bias towards the dominant output. To address this problem, we have so many over-sampling techniques to balance the data. Some of them are:

  1. Using the resample technique of pandas in python.
  2. SMOTE (Package available in python )
  3. SMOTE+TOMEK
  4. SMOTE+ENN
  5. Random Over Sampling

(For 3,4,5 in python, we need to install the imblearn package in python)

| improve this answer | | | | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.