0
$\begingroup$

Let's assume we habe an unbalanced dataset: 90% of the data belong to class A, 10% belong to class B. Furthermore, there are around as many points from class B inside of class A's cluster. Someone with a lot of expertise told me that models will weight class A more in that area.

But as far as I know, models don't just automatically weight the classes. Am I wrong? How would different models behave and why?

$\endgroup$

1 Answer 1

2
$\begingroup$

So if we take simple classification model like KNN, there are ways to handle this kind of imbalance in data. And also this kind of issues are largely seen in real world datasets.

In KNN we can use distance based weights and helping us in predicting classes. Checkout parameter weight here KNN. By default model considers uniform, but if u know u have imbalance then use weights = 'distance'.

In based classifiers as well u can see this. Check class_weight section DT_Classifier. This by default considers it as None i.e all classes have same weightage.

There are some other ways to deal with this issue,

  1. UpSample minority class
  2. Downsample majority class
  3. Use SMOTE (it creates new data based on existing points) -> model training time has impact here. SMOTE should only be used on training data never on testing data.
$\endgroup$
6
  • $\begingroup$ Hey Karthik, thank you for your answer. So what you're saying is that models don't just automatically weight the classes? Do you have more examples? One correction: The documentation for the Decision Tree says, for class_weight: "If None, all classes are supposed to have weight one." and the default is None. Therefore, the Decision Tree doesn't use a distance- or weight-based approach by default, does it? $\endgroup$ Commented Oct 30, 2022 at 9:23
  • $\begingroup$ Hey @Waschbrettwade, thanks for correcting i had misread document. I made same changes in my answer aswell. $\endgroup$ Commented Oct 30, 2022 at 10:44
  • $\begingroup$ @Waschbrettwade you need model based examples for this? or examples on approching this kind problems ? $\endgroup$ Commented Oct 30, 2022 at 10:44
  • $\begingroup$ By default model considering imbalance/ Balance depends on people who build these models. When we implement models we have to take a look at out target variables values_counts() and then make this decision. $\endgroup$ Commented Oct 30, 2022 at 10:50
  • $\begingroup$ If a model will not weight the classes itself by default, then what did the person I spoke to could have meant when he said that when the class B cluster is also filled with class A data, models will prefer class B in that area? $\endgroup$ Commented Oct 30, 2022 at 13:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.