I'm reading Cliche, Mathieu (2017). In his paper, he describes using cross entropy loss, weighted by the inverse frequency of the true classes to counteract an imbalanced dataset (this is stated on page 5, first paragraph).

Mathematically, what does this mean?

I know that cross entropy loss is defined as:

$$ -\sum_iy_i\log(\hat{y}_i) $$

where $i$ is a particular node in the output layer, $y$ is the true distribution and $\hat{y}$ is our distribution in the output layer.

How is this weighted by the inverse class frequency?

  • $\begingroup$ You might want to ask theoretical questions like this on stats.stackexchange.com, better chance to get answers $\endgroup$
    – Erwan
    Commented Sep 22, 2019 at 0:18
  • $\begingroup$ Please, add some more tags to get answers. $\endgroup$ Commented Jan 7, 2020 at 6:11


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.