1
$\begingroup$

The Probabilistic approach of logistic regression involves the MLE (Maximum Likelihood Estimation) maximizing the likelihood function, or in other words, finding the best parameters for the best fit line using partial derivatives.

The Geometric approach involves assigning +1 & -1 to the classes 1 & 0 respectively. A random line is fit to the data. The distance between the points & the line is calculated & the correct & incorrect misclassifications are identified based on the y*d formula. This process continues till we find a line with the least number of misclassifications.

Which of these approaches does logistic regression use?

$\endgroup$
1
  • $\begingroup$ Logistic regression alone does not make classifications, and the parameters, as is typical of GLMs, traditionally are estimated via maximum likelihood. $\endgroup$
    – Dave
    Nov 11, 2021 at 0:05

1 Answer 1

1
$\begingroup$

It is a common misconception that logistic regression does classification just because the $y$ variable is discrete. Logistic regression estimates the conditional probability of an event, and all $p\in(0,1)$ are fair game (arguably all $p\in[0,1]$). If one desires, one can apply a threshold to categorize the probability outputs, but this is not necessary, and there’s certainly no notion of “misclassification” in the logistic regression on its own.

As is typical of generalized linear models, the parameters of a logistic regression equation are estimated via maximum likelihood, considering the conditional response variable as Bernoulli.$^{\dagger}$ A GLM textbook like Agrest’s Foundations of Linear and Generalized Linear Models gets into the details.

Agresti, Alan. Foundations of linear and generalized linear models. John Wiley & Sons, 2015.

Vanderbilt professor Frank Harrell is a major opponent of logistic regression thresholds. Two of his blog posts get into why.

https://www.fharrell.com/post/class-damage/

https://www.fharrell.com/post/classification/

$^{\dagger}$It’s possible to consider the conditional distribution to be a binomial with more than one trial (coin flip), but this is not what you mean when you want to use logistic regression to distinguish between pictures of dogs and cats (for example). That is a binomial distribution with one trial, which is Bernoulli.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.