I know that linear regression does "regression" and logistic regression does "classification". When we implement these two methods, the only difference I could notice is the loss function: linear regression uses a loss function like mean square error and logistic uses cross-entropy. Is there any other difference that I am not aware of?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
As you have mentioned, the output of linear regression is a real value while logistic regression's represents classes(classification). Their main difference is this.
The loss function of linear regression is convex which means you always can find the optimal point using customary optimisations while if you use that for logistic regression, you may get stuck in a non-global minimum which is not optimal. Consequently, people take logarithm of that loss and call it cross entropy. For simple logistic regression tasks, it is convex.
Another difference that has to be cared about is the non-linearity which is usually applied for different tasks. For logistic regression, it is customary that people use non-linearites like Tanh or Sigmoid after the linear part, inner product of weights and inputs, for specifing the similarity to class 1 or 0 for typical binary classifications. For linear regression, people usually use Linear activation function. There is a point here. The idea of using the linear activation is not due to the need for employing a linear function. It is used because its output is not limited. Consequently, you can use other functions that are one-to-one and are not limited. Consider $y = x^3$ as an example.
answered Sep 14, 2018 at 11:23
-
$\begingroup$ Thanks for the answer. so, Can i just say that in programming, the only difference is the loss function? $\endgroup$– rawwarCommented Sep 15, 2018 at 10:00
-
$\begingroup$ The output types also can be considered. $\endgroup$ Commented Sep 15, 2018 at 18:41
$\begingroup$
$\endgroup$
Logistic Regression is used for classification.
Linear Regression is used for prediction.
Given a set of features, logistic regression uses sigmoid function to find whether it belongs to class 0 or 1. Example: Success or failure of a event can be predicted using logistic regression.
In linear regression, given a set of features, it is used to predict the value of output. The sales of a company can be found using linear regression.
Even though there is the work regression in logistic regression, it is a classification algorithm.
