# Is gradient descent useful to get the least mean squared error in linear regression?

I am new to machine learning.

I have read about the linear regression where-in the ideal model is a line which has the least mean squared error. In multi-variable linear regression we would have a plane instead of the line. However the goal is the same - least mean squared error. This is calculated as the sum of squared distance between the value and the line, divided by the number of samples.

I then read about gradient descent which is a technique to reduce the loss. So the example showed a loss function with y-axis as loss and x-axis as model params.

I'm trying to understand the relation between gradient descent and linear regression's least mean squared error.

• Related // What is your understanding about a loss function vs a regression function?
– Dave
Jul 29, 2022 at 9:57
• I'm new to this topic and when learning about linear regression they showed mean squared error as way to minimize loss. Where as the next course section said about gradient descent. So I'm not sure whether there is a relation between MSE and GD? Jul 29, 2022 at 10:06
• Mean squared error is the loss. Then you need some way to figure out the regression parameters that minimize the loss. In linear regression, we are lucky enough to have calculus give a simple equation involving matrix inversions and transposes, $\hat\beta_{OLS}=(X^TX)^{-1}X^Ty$, but other methods like logistic regression and neural networks are not so lucky and have to rely on numerical approximation like gradient descent. // If we have an equation like we do in OLS and choose to do gradient descent, gradient descent should give an answer close to that given by the explicit equation.
– Dave
Jul 29, 2022 at 10:16