0
$\begingroup$

A logistic regression involves a linear combination of features to predict the log-odds of a binary, yes/no-style event. That log-odds can then be transformed to a probability. If $\hat L_i$ is the predicted log-odds of observation $i$, then define the predicted probability of observation $i$, $\hat p_i$, by $\hat p_i=\frac{1}{1+e^{-\hat L_i}}$.

How, then, do data scientists obtain accuracy values for such models? In particular, every predicted probability is going to be greater than zero and less than one, so every prediction is going to be a little bit wrong, since binary values in this framework tend to be coded as $0$ and $1$, yet I routinely see data scientists claiming nonzero accuracy of such models.

$\endgroup$

1 Answer 1

2
$\begingroup$

You need a threshold value $t$ to assign a class based on the probability, so that if $p < t$ you assign it to class 0, and if $p >= t$ you assign it to class 1. Then, you can compute the accuracy on a test set based on the assigned class and the true class; for this, you run your model on the test dataset, get predictions and divide the number of correct predictions by the total number of predictions.

For the threshold value, by default, you can use $t = 0.5$. Alternatively, you could compute an optimal threshold on a held-out validation data set (i.e. the threshold value that maximizes the accuracy or any other metric over the said validation data).

$\endgroup$
2
  • $\begingroup$ So then the accuracy refers to something other than the logistic regression, right: the logistic regression plus the imposition of some kind of threshold? $\endgroup$
    – Dave
    Commented Feb 8, 2023 at 16:26
  • $\begingroup$ With any binary classifier that outputs the probability of belonging to the positive class, either logistic regression or any other type of model (e.g. a complex neural network) it's the same: you need a threshold to assign the end boolean result. $\endgroup$
    – noe
    Commented Feb 8, 2023 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.