0
$\begingroup$

In linear regression model, how can we define cost function. also after defining cost function how to minimize the error term?

$\endgroup$
1
$\begingroup$

Statistical programs, such as R, typicall use Least Squares estimation. It's a deterministic algorithm that makes a linear model find its optimal tuple of parameters. Because of this, you don't have to worry about the choice of a loss function.

In case you wanted to train your linear regression with a gradient descent algorithm, instead, you'd have to specify a loss function to run it. Classical loss functions for regression are: Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE).

$\endgroup$
  • $\begingroup$ Thanks Leevo, but i did not get the meaning of below in your comment. "It's a deterministic algorithm that makes a linear model find its optimal tuple of parameters. Because of this, you don't have to worry about the choice of a loss function." Does this mean that once i run LR function in R for Linear regression, in first run itself i am going to get optimal model with least error and i do not need to reduce errors further(though variable selection and transformation etc. can be further done to improve model)? $\endgroup$ – Saurabh Jan 22 at 4:31
  • $\begingroup$ Exactly. Linear regression is simple enough to allow for that. So when you run a linear regression in R you don't have to worry at all about Loss and optimization. $\endgroup$ – Leevo Jan 22 at 8:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.