0
$\begingroup$

I know that Gradient Descent is an optimization algorithm used for optimizing the cost of the loss function.

Does Linear Regression model of the sklearn package use Gradient Descent ?

Or it simply uses normal equations of curve fitting (which I read in my 11th class) to find the slope and the coefficient(s)?

$\endgroup$

2 Answers 2

0
$\begingroup$

No, Linear Regression typically uses a method called Ordinary Least Squares (OLS) to find the best fit line. OLS minimizes the sum of the squared residuals (the differences between the observed and predicted values) to find the optimal parameters for the regression line.

However, when the dataset is large and cannot fit into memory, or when the problem is a part of a larger optimization problem, variants of Linear Regression such as Stochastic Gradient Descent (SGD) or Mini-Batch Gradient Descent can be used, which are iterative optimization algorithms.

$\endgroup$
0
$\begingroup$

According to the documentation:

From the implementation point of view, this is just plain Ordinary Least Squares (scipy.linalg.lstsq) or Non Negative Least Squares (scipy.optimize.nnls) wrapped as a predictor object.

So no, it does not perform Gradient Descent.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.