0
$\begingroup$

I am going through this course. The professor is talking about the logistic regression cost function:

$$ P(y|x) = \hat y^y (1- \hat y)^{(1-y)} $$

Taking log on both sides provides:

$$ \begin{align} log[P(y|x)] &= log[\hat y^y (1- \hat y)^{(1-y)}]\\ &= - L(\hat y,y) \end{align} $$

Why does the logistic regression cost function need to be the negative of log?

$\endgroup$
2
$\begingroup$

Normally we want to maximize the likelihood (consequently the log likelihood). This is the reason why we call this method maximum likelihood estimation. We want to determine the parameters in such a way that the likelihood (or log likelihood) is maximized.

When we think about the loss function we want to have something that is bounded by 0 from below and is unbounded for positive values. Our goal is to minimize the cost function. Hence, we take the negative of the log likelihood and use it as our cost function.

It is important to note that this is just a convention. You could also take the log likelihood and maximize it, but in this case, we would not be able to interpret it as a cost function.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.