Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
Still I have no clear concept when I should choose linear OR Logistic regression.
In addition, when I can say either linear or logistic regression performing better?
In particular, I want to know When should I choose Linear Regression and When to choose Logistic regression?
Linear Regression is used for predicting continuous variables.
Logistic Regression is used for predicting variables which has only limited values.
Let me quote a nice example which can help you make the difference between the both:
For instance, if X contains the area in square feet of houses, and Y contains the corresponding sale price of those houses, you could use linear regression to predict selling price as a function of house size. While the possible selling price may not actually be any, there are so many possible values that a linear regression model would be chosen.
If, instead, you wanted to predict, based on size, whether a house would sell for more than 200K, you would use logistic regression. The possible outputs are either Yes, the house will sell for more than $200K, or No, the house will not.
My two cents...
Not 100% accurate, but can give you a rough idea...
Linear regression makes a linearization of a problem where $y = f(x)$, with $x$ and $y$ are continuous variables.
Now imagine that you want to predict a kind of boolean behavior (yes/no) based on a $x$ value. For example, based on your salary, are you happy or not.
You can say happy = 1 and not happy = 0. You can make a scatter plot with all pairs (salary, happy) (happy in the vertical axis).
You can try to make a line to separate the happy people from the unhappy ones, but you'll see quickly that's not working well (what is a value in the middle, etc.).
A better idea would be to draw a kind of s curve which will pass as best through the points you have.
That's what the logistics regression makes. It basically makes linear the s curve by transforming the $y$ values: this is the Logit function.
We then say that we predict "True" if the predicted logit is higher than a threshold. This threshold corresponds often to 0.5, which is the inflexion point of the curve (some tools are only allowing to use 0.5). That threshold is in fact the probability.
When we have more than yes/no possibilities, one solution is to make the logistics regression for all the possibilities (e.g. if you have A, B, C, that will be A/not A, B/not B, C/not C) and take the possibility that give you highest probability. This is called the "one-vs-all" approach.
