I'm new to machine learning and willing to study and work with machine learning. It just that I still don't get to understand the benefits of using the normal equation in some occasion in comparison with gradient descent. I use Andrew Ng's course on Coursera but the notation really makes me a hard time to understand.

I want to know more about the derivation of the cost function $J(\theta)$ for polynomial regression and the reason why he uses the transpose of vector $x(i)$

  • 2
    $\begingroup$ Andrew Ng has more than one course on Coursera, and uses subtly different notation depending on the course and topic. Please could you provide more details - the full equations you are asking about and a link to the specific lecture would be great. $\endgroup$ Commented Aug 1, 2018 at 16:44
  • $\begingroup$ You need to delete the questioner at SO, if you are going to ask here. Thanks. stackoverflow.com/q/51624899/7311767 $\endgroup$
    – Stephen Rauch
    Commented Aug 1, 2018 at 18:06
  • $\begingroup$ @StephenRauch is that violating something? $\endgroup$ Commented Aug 1, 2018 at 18:28
  • 1
    $\begingroup$ @Media, Yes it is... Cross posting is strongly discouraged... meta.stackexchange.com/q/64068/347908 $\endgroup$
    – Stephen Rauch
    Commented Aug 1, 2018 at 18:30
  • $\begingroup$ Ok I have already deleted the post in SO. It is in week 2 normal equation, the equation how to use the ϴ=(X^TX)^(-1)X^Ty or this link $\endgroup$
    – aarnphm
    Commented Aug 1, 2018 at 18:58

1 Answer 1


You want to solve $X \times \theta = Y$. Actually, you have to find the parameters $\theta$. To find it, you should multiply both sides by $X^{-1}$ but it may have not an inverse. Consequently, you multiply each side by $X^t$ due to the fact that $X^tX$ has the inverse. After that, you multiply each side by $(X^tX)^{-1}$ which leads to a closed form equation for finding $\theta$. It should be mentioned that $X^tX$ does not always have an inverse. If you want to be sure it has it, you have to exploit data samples which are more than the number of features and they should not be linearly dependent otherwise the constructed matrix will not inverse and you will not be able to use it.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.