I'm handling a high imbalanced dataset, thus, I'm weighing the loss function in order to penalize the misclassification of the minority classes, I set the weights in each batch as follows:
w[i] = num_total_instances / num_instances_of_class[i]
Where 'i' goes from 0 to N-1 (for N classes), 'num_total_instances' is the total number of instances in the batch, and 'num_instances_of_class[i]' is the number of instances of class 'i' in the batch.
The problem is, as I'm doing it on each batch, It may occur that, in a certain batch there is no instance of class 'j', thus w[j] = inf. ¿What weigh should I set for class 'j'? At this moment, I'm setting this w[j] to 1E6
I know that there are other approaches to fight against imbalanced datasets and I also know that I can pre-compute the weights for the whole dataset and use them (fixed) cross all batches, but, I'd like the specific answer for this question, whereas it makes sense. For sure, it will be well received other suggestions for the calculation of the weights.