Suppose for a single training example, the true label is [1 0 0 0 0] while the predictions be [0.1 0.5 0.1 0.1 0.2]. How to calculate its binary cross entropy? And how output value decides whether a sample belongs to class a or b? I also want to know that that in the formula of binary cross entropy what is 1-p?


1 Answer 1


Binary cross-entropy is a simplification of the cross-entropy loss function applied to cases where there are only two output classes. Essentially it can be boiled down to the negative log of the probability associated with your true class label.

Cross entropy

Cross entropy is defined as

$L = -\sum ylog(p)$

where $y$ is the binary class label, 1 if the correct class 0 otherwise. And $p$ is the probability of each class.

Let's look at an example, if for an instance $X$ the output label is 0 and your model output was $[0.7, 0.3]$. Then we can see that the loss function using binary cross entropy is

$L = -(1*log(0.7) + 0*log(0.3)) = 0.155$

Notice how the loss is only affected by the true label's prediction probability.

Binary cross-entrop

Binary cross entropy is for an example with 2 output classes and can be simplified as

$L = (ylog(p) + (1-y)log(1-p)) = (1*log(0.7) + (1-1)log(1-0.7)) = 0.155$

Your example

For your example, $[1,0,0,0,0]$ and predictions $[0.1 0.5 0.1 0.1 0.2]$ the loss is

$L = -(1*log(0.1) + 0*log(0.5) + 0*log(0.1) + 0*log(0.1) + 0*log(0.2)) = 1$

  • $\begingroup$ ok So what the higher value of L tells us? Does it gives any information about the two classes or about our model?I have to use this in machine learning $\endgroup$
    – herry
    Oct 1, 2018 at 7:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.