1
$\begingroup$

I am learning linear regression right now. In the most of the examples of implementation of this method, which I found, gradient descent is used.

Is there a better way to optimize linear regression than gradient descent?

$\endgroup$
2
4
$\begingroup$

Is there a better way to optimize linear regression than gradient descent?

If by better you mean finding the better separator, no you can't due to the fact that the cost function for linear regression is convex which means there is just one optimal point.

If you want to optimize using different algorithms, there are different kinds of solutions. Gradient-based algorithms like Adam and RMSProp are of those. You also can use normal equation.

$\endgroup$
4
  • 1
    $\begingroup$ I would consider this an incomplete answer. This solution works only through the perspective of neural networks. For a typical problem, gradient descent would be outperformed by other solutions like QR decomposition or least squares estimation. $\endgroup$
    – Jon
    Jul 8 '18 at 23:33
  • $\begingroup$ I am learning linear regression right now. In the most of the examples of implementation of this method .... The question is about something else, you could add extra details but they do not have any relation to the question. $\endgroup$
    – Media
    Jul 9 '18 at 1:11
  • $\begingroup$ You might want to review linear regression. The role of gradient descent is for parameter estimation. For parameter estimation there are already well established methods that outperform in most problems. GD is not guaranteed to land on the unbiased estimation, where other methods almost always will (as long as you have >30 samples). $\endgroup$
    – Jon
    Jul 9 '18 at 1:52
  • $\begingroup$ What do you mean by outperform? Linear regression just has one optimal point. By using second order optimizations you can accelerate its convergance. $\endgroup$
    – Media
    Jul 9 '18 at 1:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.