# Cost function - Log Loss query

What is the purpose of using "log" in the logistic regression cost function "log loss"?

• A full answer should explain why this is the case (and I know it’s shown somewhere on the statistics Stack), but minimizing that loss function is equivalent to maximum likelihood estimation of the logistic regression parameters.
– Dave
Nov 8, 2021 at 17:22

Let's explore the use case for binary classification. In binary classification the labels are drawn from Bernoulli distribution. For each example the likelihood of the Bernoulli distribution is

$$p^y*(1-p)^{(1-y)}$$.

We want to maximize the likelihood of the entire dataset, which means we want to maximize the product of all the examples.

Because we want it to be convenient for the optimizer we do two things:

1. We want a minimization problem, so we minimize the negative likelihood instead of maximizing likelihood.
2. We convert from a product to a sum (which is much easier to get derivatives) In order to do it we apply log transformation which is monotonic transformation which promises us that the optimal parameters will remain the same i.e. minimizing the log likelihood is equivalent for minimizing the likelihood.

Log loss is the negative log likelihood. Taking the log of a single example we get:

$$y_i \log(p_i) + (1-y_i) \log (1-p_i)$$.

The log loss is just the negative sum of all the examples.