I'm handling a high imbalanced dataset, thus, I'm weighing the loss function in order to penalize the misclassification of the minority classes, I set the weights in each batch as follows:

w[i] = num_total_instances / num_instances_of_class[i]

Where 'i' goes from 0 to N-1 (for N classes), 'num_total_instances' is the total number of instances in the batch, and 'num_instances_of_class[i]' is the number of instances of class 'i' in the batch.

The problem is, as I'm doing it on each batch, It may occur that, in a certain batch there is no instance of class 'j', thus w[j] = inf. ¿What weigh should I set for class 'j'? At this moment, I'm setting this w[j] to 1E6

I know that there are other approaches to fight against imbalanced datasets and I also know that I can pre-compute the weights for the whole dataset and use them (fixed) cross all batches, but, I'd like the specific answer for this question, whereas it makes sense. For sure, it will be well received other suggestions for the calculation of the weights.

  • $\begingroup$ Did I answer it? $\endgroup$ – David Masip May 16 '18 at 10:08
  • $\begingroup$ Yes, sure. I voted it up $\endgroup$ – ignatius May 16 '18 at 10:10
  • $\begingroup$ You can also accept it if you are satisfied with this answer. $\endgroup$ – David Masip May 16 '18 at 10:18
  • $\begingroup$ Sure, I can also wait for more answers. If no more answers are received, or no further answers improve yours, I'll accept it $\endgroup$ – ignatius May 16 '18 at 10:33

I don't think that the weight you use matters, whether you set it to $10^6$ or $10^8$. This is because, as examples of that class are not in the batch, the loss function for that batch will not have any of those examples contributing, so that weight won't appear. If something does not appear in the loss function, it is not used at all. For that reason, it doesn't really matter what weight to use.

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.