1
$\begingroup$

Hi I am new to Linear Regression. I want to know

what is the difference b/w Gradient Descent and Mean Square Error in Linear Regression using machine learning?

And

When to use Gradient Descent and Mean Square Error in Linear Regression using machine learning? Or

When to use which algorithm in Linear Regression.?

Can anyone explain.?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Mean Square Error is a loss function not an algorithm. You don't need to choose between MSE and Gradient Descent. Did you perhaps mean to name a different algorithm (e.g. the Normal Equation)? $\endgroup$
    – Neil Slater
    Commented Oct 4, 2018 at 7:56

2 Answers 2

Reset to default
2
$\begingroup$

To train a model, two processes have to be followed. From the predicted output, the error has to be calculated w.r.t the real output. Once the error is calculated, the weights of the model has to be changed accordingly.

Mean square error is a way of calculating the error. Depending upon the type of output, the error calculation differs. There are absolute errors, cross-entropy errors, etc. The cost function and error function are almost the same.

Gradient descent is an optimization algorithm or simply update rule, used to change the weight values. Some of the variations are Stochastic gradient descent, momentum, AdaGrad, AdaDelta, RMSprop, etc. More about Optimization algorithms

Improve this answer
$\endgroup$
2
  • $\begingroup$ :- Hi Can you please tell me when to use Gradient Descent in Linear Regression..? Because their might be different algorithm .? Why only Gradient Descent.? Is their any specific reason for using Gradient Descent.? $\endgroup$
    – Sanjiv
    Commented Oct 4, 2018 at 11:11
  • $\begingroup$ Go through the link provided. Optimization algorithms are mostly about quickly reaching the minimum point. There are multiple ways to reach it. There is no strong rule which is the best one. Basically gradient descent is obtaining the slope of the cost function and going in the negative direction, so as to minimize the cost. $\endgroup$
    – chmodsss
    Commented Oct 4, 2018 at 11:53
1
$\begingroup$

Gradient Descent is an algorithm for minimizing some functions like the Mean Squared Error function. As described, GD takes iterative steps downhill in the negative gradient direction to find the min. Normal Equations are a way of directly solving the quadratic eqn for the weights that minimize the MSE. But solving Normal Eqns involves inverting a matrix which is of size N_features x N_features, an O(N_features**3) operation. So when N_features > 1000 or so it starts to slow while GD is O(N).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.