Unbalanced Classification: What happens when many points of the bigger class are inside of the smaller class' area?

Question

Let's assume we habe an unbalanced dataset: 90% of the data belong to class A, 10% belong to class B. Furthermore, there are around as many points from class B inside of class A's cluster. Someone with a lot of expertise told me that models will weight class A more in that area.

But as far as I know, models don't just automatically weight the classes. Am I wrong? How would different models behave and why?

Answer 1

So if we take simple classification model like KNN, there are ways to handle this kind of imbalance in data. And also this kind of issues are largely seen in real world datasets.

In KNN we can use distance based weights and helping us in predicting classes. Checkout parameter weight here KNN. By default model considers uniform, but if u know u have imbalance then use weights = 'distance'.

In based classifiers as well u can see this. Check class_weight section DT_Classifier. This by default considers it as None i.e all classes have same weightage.

There are some other ways to deal with this issue,

1. UpSample minority class
2. Downsample majority class
3. Use SMOTE (it creates new data based on existing points) -> model training time has impact here. SMOTE should only be used on training data never on testing data.