# Minimizing cross entropy vs minimizing negative probabilities

1. The cross entropy loss can be written as $$L_1 = -\sum_i\sum_c y_{ic}\log P_{ic}$$, where $$i$$ represents images and $$c$$ are the classes. $$y_{ic}=1$$ for the correct class.
2. Instead of L_1 I can minimize the following $$L_2 = -\sum_i\sum_c y_{ic}P_{ic}$$

When I use these as the loss functions to CNN I found that for a particular problem $$L_2$$ performs significantly better than $$L_1$$. However, I am unable to explain it. What could be the reason for this?

• Could you please confirm that the loss function in 2 is what you are referring to as L2 – mahesh ghanta Jun 19 '19 at 4:04
• Also please share distribution of 0s and 1s in your dataset if that's ok – mahesh ghanta Jun 19 '19 at 4:06
• I am using a weighted loss. I have 6 classes. So we can assume that all the classes contains almost equal number of images. – user570593 Jun 19 '19 at 5:50
• In second case we are just making the value of negative as much as possible and solving it. The only reason I can think of is that we are penalizing the error with the true P value in L2 which is between 0 and 1 in this case vs 0 and infinity in the case of L1. My assumption is if any class is under represented L1 cannot wrongly predict that class still but L2 can ignore that class. – mahesh ghanta Jun 19 '19 at 8:13
• It's an interesting problem. Please keep me posted if you find a reason. – mahesh ghanta Jun 19 '19 at 8:22