1
$\begingroup$

Suppose you have a dataset with m = 50 examples and n = 15 features for each example. You want to use multivariate linear regression to fit the parameters theta to our data. Should you prefer gradient descent or the normal equation and why?

$\endgroup$
  • $\begingroup$ If this is a question from an online course, you should bear in mind whether you have signed the "honour agreement" that all the work is your own. Getting an answer by passing the question onto a Q&A site will not count as your own work, and if this does apply to you, it may cause any certificates you gain as a result to be invalidated at a later date. $\endgroup$ – Neil Slater Oct 11 '19 at 14:33
  • $\begingroup$ To avoid this, you should ask a different, related question in your own words. E.g. "When should I prefer to use the normal equation rather than gradient descent, for linear regression?". If you then understand the answers given, and that makes you select the correct result in the online quiz, then that is your own work, and better for your learning too! $\endgroup$ – Neil Slater Oct 11 '19 at 14:35
1
$\begingroup$

You need to think in terms of sample size.
You don't have enough examples to use Gradient Descent. I believe you need at least several 100s.
Conversely, doing inverse and transpose on small matrices like this with Normal Equation should be pretty fast.

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Actually both gradient descent and normal equation should give the same result on the example dataset (unless it has problems, such as many x rows in X being the same, but then both will fail but in different ways). There is no need for extra records here in order to use gradient descent, both GD and normal equation should give the same results. $\endgroup$ – Neil Slater Oct 11 '19 at 14:41
1
$\begingroup$

If you can, it is preferable to use the normal equation to estimate the coefficients for multivariate linear regression. Since the normal equation is a closed-form expression, it will be faster than gradient descent.

Given you have relatively few examples and features, inverting the matrix is not an issue.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.