1
$\begingroup$

I'm completing a DataCamp course where we are introduced to the log loss formula for binary classification:

log loss formula for binary classification

Two scenarios are given to show how the formula is used. One with p=0.1 and one with p=0.5. The answers the instructor displayed were 2.3 and .69, respectively. However, using a calculator, the answers for log(0.1) and log(0.5) are -1 and -0.30, respectively. I later tried using natural log instead and got the same answers as the instructor, except negative. Specifically, the calculator returned -2.3 for ln(0.1) and -0.69 for ln(0.5).

Is it common in math for log to implied to be "ln" or "log e" without stating it explicitly in the formula? Also, is there something about the log loss binary classification formula that suggest that the absolute value of the result should be taken?

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Back when I studied math, $log$ was always assumed to be the natural logarithm unless stated otherwise. The less applied the field the more likely that is to be true I'd guess ;-) $\endgroup$ – oW_♦ Aug 5 '19 at 23:35
  • $\begingroup$ @oW_ Amont mathematicians, $\ln$ is used to denote the natural logarithm, whereas an undecorated $\log$ refers to the logarithm with base 10. This varies by discipline though. For instance, computer scientists commonly use $\lg$ or $\log$ to denote the logarithm with a base of 2. $\endgroup$ – Andrew Maurer Aug 6 '19 at 8:27
4
$\begingroup$

This comes down to the change-of-base formula. For any two numbers $a$ and $b$, the following equation is true.

$$ \log_a(x) = \frac{\log_b(x)}{\log_b(a)}. $$

What this means is that the errors are proportional. So if you wanted to change to using $\log_{10}$, you would simply end up multiplying by a constant factor, and model selection would be the same.

Explicitly,

$$logloss(N=1) = \frac{y \log_{10}(p) + (1 - y) \log_{10}(p)}{\log_{10}(e)}$$

Or, equivalently

$$ logloss(N=1) \cdot \log_{10}(e) = y \log_{10}(p) + (1 - y) \log_{10}(p) $$

In other words: The base of the logarithm doesn't matter because everything ends up being proportional.

Improve this answer
$\endgroup$
2
  • $\begingroup$ So was the formula I posted written incorrectly? It's a screenshot from the lesson. $\endgroup$ – Mountain Scott Aug 5 '19 at 21:24
  • 1
    $\begingroup$ @MountainScott It was written correctly, but as you noticed the textbook you're using uses the natural logarithm. I was explaining how it relates to $\log_{10}$. $\endgroup$ – Andrew Maurer Aug 6 '19 at 8:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.