12
$\begingroup$

I have 3 classes with this distribution:

Class 0: 0.1169
Class 1: 0.7668
Class 2: 0.1163

And I am using xgboost for classification. I know that there is a parameter called scale_pos_weight.

But how is it handled for 'multiclass' case, and how can I properly set it?

$\endgroup$
12
$\begingroup$

scale_pos_weight is used for binary classification as you stated. It is a more generalized solution to handle imbalanced classes. A good approach when assigning a value to scale_pos_weight is:

sum(negative instances) / sum(positive instances)

For your specific case, there is another option in order to weight individual data points and take their weights into account while working with the booster, and let the optimization happen regarding their weights so that each point is represented equally. You just need to simply use:

xgboost.DMatrix(..., weight = *weight array for individual weights*)

You can define the weights as you like and by doing so, you can even handle imbalances within classes as well as imbalances across different classes.

$\endgroup$
4
$\begingroup$

This answer by @KeremT is correct. I provide an example for those who still have problems with the exact implementation.

weight parameter in XGBoost is per instance not per class. Therefore, we need to assign the weight of each class to its instances, which is the same thing.

For example, if we have three imbalanced classes with ratios

class A = 10%
class B = 30%
class C = 60%

Their weights would be (dividing the smallest class by others)

class A = 1.000
class B = 0.333
class C = 0.167

Then, if training data is

index   class
0       A
1       A
2       B
3       C
4       B

we build the weight vector as follows:

index   class    weight
0       A        1.000
1       A        1.000
2       B        0.333
3       C        0.167
4       B        0.333
$\endgroup$
3
$\begingroup$

Everyone stumbles upon this question when dealing with unbalanced multiclass classification problem using XGBoost in R. I did too!

I was looking for an example to better understand how to apply it. Invested almost an hour to find the link mentioned below. For all those who are looking for an example, here goes -

https://datascience.stackexchange.com/a/9493/37156

Thanks wacax

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.