0
$\begingroup$

I'm handling a high imbalanced dataset, thus, I'm weighing the loss function in order to penalize the misclassification of the minority classes, I set the weights in each batch as follows:

w[i] = num_total_instances / num_instances_of_class[i]

Where 'i' goes from 0 to N-1 (for N classes), 'num_total_instances' is the total number of instances in the batch, and 'num_instances_of_class[i]' is the number of instances of class 'i' in the batch.

The problem is, as I'm doing it on each batch, It may occur that, in a certain batch there is no instance of class 'j', thus w[j] = inf. ¿What weigh should I set for class 'j'? At this moment, I'm setting this w[j] to 1E6

I know that there are other approaches to fight against imbalanced datasets and I also know that I can pre-compute the weights for the whole dataset and use them (fixed) cross all batches, but, I'd like the specific answer for this question, whereas it makes sense. For sure, it will be well received other suggestions for the calculation of the weights.

Improve this question
$\endgroup$
4
  • $\begingroup$ Did I answer it? $\endgroup$
    – David Masip
    Commented May 16, 2018 at 10:08
  • $\begingroup$ Yes, sure. I voted it up $\endgroup$
    – ignatius
    Commented May 16, 2018 at 10:10
  • $\begingroup$ You can also accept it if you are satisfied with this answer. $\endgroup$
    – David Masip
    Commented May 16, 2018 at 10:18
  • $\begingroup$ Sure, I can also wait for more answers. If no more answers are received, or no further answers improve yours, I'll accept it $\endgroup$
    – ignatius
    Commented May 16, 2018 at 10:33

1 Answer 1

Reset to default
2
$\begingroup$

I don't think that the weight you use matters, whether you set it to $10^6$ or $10^8$. This is because, as examples of that class are not in the batch, the loss function for that batch will not have any of those examples contributing, so that weight won't appear. If something does not appear in the loss function, it is not used at all. For that reason, it doesn't really matter what weight to use.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.