I am trying to use deviance in order to optimize my network. Note that my $y_{true}$ values are equal to 0 more than in 90% of cases (can also be 1 or 2).

The deviance is calculated as : $2 \times (log(y_{true}) - log(y_{pred}))$

The problem in my case is that $log(y_{true})$ when $y_{true}= 0$ gives me $-\infty$.

I derivated the deviance formula in my case (poisson unscaled deviance) and got this result :

  • If $y_{true} = 0$, then deviance is : $2D \times y_{pred}$ where $D$ is a feature of my data.
  • If $y_{true} \ne 0$, then deviance is : $2D \times (y_{true} \times ln(y_{true}) - y_{true} \times ln(y_{pred}) - y_{true} + y_{pred})$

This is the loss i came up with :

def custom_loss(data, y_pred):

    y_true = data[:, 0]
    d = data[:, 1:]
    # condition
    mask2 = keras.backend.not_equal(y_true, 0) #i.e. y_true != 0
    mask2 = KB.cast(mask2, KB.floatx())
    # returns 0 when y_true =0, 1 otherwise
    #calculate loss using d...
    loss_value = 2 * d * y_pred + mask2 * (2 * d * y_true * KB.log(y_true) + 2 * d * y_true * KB.log(y_pred) - 2 * d * y_true)
    return loss_value

Using a mask in order to only compute logs when $y_{true}$ is different from 0. I don't think the formula is wrong but it still gives me NaN as loss result...

If anyone knows what is the matter here, or knows about a way to get around this problem i'd be very thankful.



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.