4
$\begingroup$

Still I have no clear concept when I should choose linear OR Logistic regression.
In addition, when I can say either linear or logistic regression performing better?

In particular, I want to know When should I choose Linear Regression and When to choose Logistic regression?

$\endgroup$
  • $\begingroup$ @Sean Owen, In particular, I want to know When should I choose Linear Regression and When to choose Logistic regression? $\endgroup$ – Masudur Dec 13 '15 at 12:03
  • $\begingroup$ The difference is obviously the dependent variable (as stated in the answer). $\endgroup$ – HelloWorld Dec 14 '15 at 1:11
5
$\begingroup$

Linear Regression is used for predicting continuous variables.

Logistic Regression is used for predicting variables which has only limited values.

Let me quote a nice example which can help you make the difference between the both:

For instance, if X contains the area in square feet of houses, and Y contains the corresponding sale price of those houses, you could use linear regression to predict selling price as a function of house size. While the possible selling price may not actually be any, there are so many possible values that a linear regression model would be chosen.

If, instead, you wanted to predict, based on size, whether a house would sell for more than 200K, you would use logistic regression. The possible outputs are either Yes, the house will sell for more than $200K, or No, the house will not.

$\endgroup$
  • $\begingroup$ Is not the logistic regression like classification of two class (In case of your example) ? $\endgroup$ – Masudur Dec 13 '15 at 12:00
  • $\begingroup$ In this case, Yes. But, in general the difference is simply that: The dependent variable of linear regression is continuous and that of logistic regression is categorical $\endgroup$ – Dawny33 Dec 13 '15 at 12:38
1
$\begingroup$

My two cents...

Not 100% accurate, but can give you a rough idea...

Linear regression makes a linearization of a problem where $y = f(x)$, with $x$ and $y$ are continuous variables.

Now imagine that you want to predict a kind of boolean behavior (yes/no) based on a $x$ value. For example, based on your salary, are you happy or not.

You can say happy = 1 and not happy = 0. You can make a scatter plot with all pairs (salary, happy) (happy in the vertical axis).

You can try to make a line to separate the happy people from the unhappy ones, but you'll see quickly that's not working well (what is a value in the middle, etc.).

A better idea would be to draw a kind of s curve which will pass as best through the points you have.

That's what the logistics regression makes. It basically makes linear the s curve by transforming the $y$ values: this is the Logit function.

We then say that we predict "True" if the predicted logit is higher than a threshold. This threshold corresponds often to 0.5, which is the inflexion point of the curve (some tools are only allowing to use 0.5). That threshold is in fact the probability.

When we have more than yes/no possibilities, one solution is to make the logistics regression for all the possibilities (e.g. if you have A, B, C, that will be A/not A, B/not B, C/not C) and take the possibility that give you highest probability. This is called the "one-vs-all" approach.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.