0
$\begingroup$

I am looking for methods that use sample weighting instead of class weighting for the imbalanced classification. I think sample weighting is more precise than weighting all the samples from one class with the same weight. Am I thinking right? Is there any paper that discusses this issue?

$\endgroup$
1
  • 2
    $\begingroup$ I don't see how the instances would be weighted? When using a weight by class one can counter the effect of class imbalance, so what would be the point of having different weights for different instances in the same class? $\endgroup$
    – Erwan
    Apr 5, 2021 at 23:17

2 Answers 2

3
$\begingroup$

If you assign to each instance the weight of the corresponding class the effect will be similar and sometimes exact.

I say similar because there are methods which uses sample weighting intrinsically, for example AdaBoost or almost all the types of decision trees, but they do perform also some arithmetics with those weights which can lead to different results. For example in CART trees instances have a weight and if it happens that at a test node to have a missing value, the same instance goes into both child nodes but with weights splitted (their sum is equal with the original weight). So instace weights could have a more granular effect, and perhaps a better resolution and in the end maybe a better performance.

A second way often encountered in modelling is to introduce learning bias into the problem. Take for example weighted linear regression. The learning bias is your knowledge about how much you trust the value of an observation. It can be done automatically for example judging how much variability has the observation relatively to it's neighbour observations (high variance means less trustful value). The same idea of encoding trust into observations could be pushed into any classifier if it allows you to weight instances.

A last note is that a common interpretation of class weighting is in terms of cost of error (how much it will hurt you if a class A instances is predicted as class B instance) and instance weighting is usually encoded like how much trust do you have in an observation or, alternatively, how much do you want that that observation and similar with that one to be correctly classified. And, as I said in the beggining, they can have a similar effect.

Class imbalance is just a particular problem to be solved, instance weighting allows you to solve this problem and also other issues related with your data.

$\endgroup$
1
  • $\begingroup$ Thanks a lot for your complete answer. Today I came back to read again the answer, and I got one new question. What do you mean that instance weighting can also solve "other issues"? I thought about it but nothing came to my mind. Would you name some other issues that instance weighting might be helpful for them? $\endgroup$
    – user137927
    Apr 26, 2021 at 22:02
1
$\begingroup$

I would like to add that fairly recently there has been a reinforcement learning implementation that rewards the classifier fully when labelling the minority class whereas it would be partially for the majority class. You can find it by the name Deep Reinforcement Learning for Imbalanced Classification. Not sure if that's something that you were looking for, but there you go.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.