I have a dataset with 4 classes, say their distribution in the training-set is

$P_{prior}(C1) = 60\% $

$P_{prior}(C2) = 25\% $

$P_{prior}(C3) = 10\% $

$P_{prior}(C4) = 5\% $

I have trained a CNN over the data set (no resampling i.e the network is trained over the true distribution), and I now want to predict unseen data. Usually we assign the data to the class with the highest score/probability, but say I get an outcome from the network on the unseen data, $X$ as:

$P(C1) = 50\%$,

$P(C2) = 10\%$

$P(C3) = 10\%$


we here notice that the probability of C4 is 6 timers greater than it being C4 at random, and C1 has lower probability than than being C1 at random. Thus I would suggest setting the label as C4 even though $P(C1)>P(C4)$.

My question is; is that apporach wrong, do we always just pick the highest score or does it make sense to scale each outcome by their prior-distribution i.e setting

$\tilde{P}(C1)=P(C1)/P_{prior}(C1) = 0,83$

$\tilde{P}(C2)=P(C2)/P_{prior}(C2) = 0,4$

$\tilde{P}(C3)=P(C3)/P_{prior}(C3) = 1 $

$\tilde{P}(C4)=P(C4)/P_{prior}(C4) = 6 $

and now assign the label here with the greatest score i.e $C4$ (in case of a tie, pick the greater $P(Ci)$)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.