2
$\begingroup$

I have a regression problem where the output y is a single probability, i.e. real number that varies in the interval [0, 1]

While using L1 or L2 loss will very likely work well, I feel that they are not the most appropriate options considering that the range [0, 1] is already well defined.

Is Binary Cross Entropy (BCE Loss in pytorch) the most appropriate in this case?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Not sure what you are doing exactly, but you may have a look at beta regression datascience.stackexchange.com/a/57686/71442 $\endgroup$
    – Peter
    Aug 31, 2020 at 20:28
  • $\begingroup$ By L1 loss, do you mean "sum of the all the absolute differences between the true value and the predicted value" or lasso regression? $\endgroup$
    – Brian Spiering
    Jul 19, 2021 at 15:56

2 Answers 2

Reset to default
0
$\begingroup$

At first I was going to say:

It doesn't make sense to use use cross entropy loss in a regression problem!

See explanation here.

But then I realised that if you are really trying to do regression on probabilities it could have some sense.

But still, why would you use it instead of L1, L2? So maybe try it and let me know if it works better!

Improve this answer
$\endgroup$
1
  • $\begingroup$ Well.. that is exactly what the question is about. :) Normally you would not use it, however, here it might be appropriate. I am looking for an explanation that is not based on experimentation. $\endgroup$
    – Juan Leni
    Oct 1, 2018 at 12:40
0
$\begingroup$

Predicting probabilities is can be framed as a beta regression.

That is a separate issue than adding a regularization term (i.e., L1 or L2).

Improve this answer
$\endgroup$
1
  • $\begingroup$ I got the feeling that the OP mentioned L1 and L2 as MAE and MSE loss functions, not as regularization. $\endgroup$
    – Dave
    Jul 19, 2021 at 15:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.