0
$\begingroup$

When we have a binary classification problem, we use a sigmoid activation function in the output layer+ a binary crossentropy loss. We also need to one hot encode the target variable.This s a binary classification problem meaning that we can have some samples with y_pred=0. As we know the binary crossentropy loss has the quantity log(y_pred), this means that we can have log(0) for samples that belong to class 0. However, log(0) is undefined ! So how the binary crossentropy is still computable ??? The same can be asked for softmax+ categorical crossentropy for multiclassification problems.

$\endgroup$

1 Answer 1

1
$\begingroup$

Adding a value equal to the machine epsilon will give you an offset in log that is large enough to not change your result but will give you an expression that you can evaluate:

import numpy as np
print(0*np.log(0))
print(0*np.log(0+np.finfo(float).eps))
nan
-0.0
$\endgroup$
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.