1
$\begingroup$

So based on the website:

https://towardsdatascience.com/understanding-binary-cross-entropy-log-loss-a-visual-explanation-a3ac6025181a

The binary cross entropy loss is defined as:

enter image description here

Which is applicable for output range {0,1}. However, what if i scale the output to be now {-1,1} instead? How would the new cross entropy loss be derived?

$\endgroup$
3
$\begingroup$

One thing you can do, is forcing your labels (${-1,1}$) to be ${0,1}$ using this simple linear transformation:

\begin{equation*}\hat{y} = (y + 1) / 2\end{equation*}

This way, -1 maps to 0, and 1 maps to 1.

For practical purposes, you can either change the outputs and labels of your model directly (before applying the original BCE), or slightly change to BCE loss function according to the linear transformation:

\begin{equation}H_p(q) = -\frac{1}{N} \sum_{i=1}^N \frac{(y_i+1)}{2} \log\left(p\left(y_i\right)\right) + (1-\frac{(y_i+1)}{2})\log\left(1-p\left(y_i\right)\right)\end{equation}

Notice that I changed only the coefficients, assuming that the probability function $p$ only returns meaningful values for ${-1,1}$

NOTE: obviously, all of this is relevant only for the binary cross entropy case.

$\endgroup$
0
$\begingroup$

Another option that u can do is instead (1-yi) one can use (-1-yi) in cross-entropy formula.

Hp(q)=−1/N ∑i=1N(yi)log(p(yi))+(-1-yi)log(1−p(yi)).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.