Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I am going through this course. The professor is talking about the logistic regression cost function:
$$ P(y|x) = \hat y^y (1- \hat y)^{(1-y)} $$
Taking log on both sides provides:
$$ \begin{align} log[P(y|x)] &= log[\hat y^y (1- \hat y)^{(1-y)}]\\ &= - L(\hat y,y) \end{align} $$
Why does the logistic regression cost function need to be the negative of log?
Normally we want to maximize the likelihood (consequently the log likelihood). This is the reason why we call this method maximum likelihood estimation. We want to determine the parameters in such a way that the likelihood (or log likelihood) is maximized.
When we think about the loss function we want to have something that is bounded by 0 from below and is unbounded for positive values. Our goal is to minimize the cost function. Hence, we take the negative of the log likelihood and use it as our cost function.
It is important to note that this is just a convention. You could also take the log likelihood and maximize it, but in this case, we would not be able to interpret it as a cost function.
