# What is the differences between normal equation and gradient descent for polynomial regression

I'm new to machine learning and willing to study and work with machine learning. It just that I still don't get to understand the benefits of using the normal equation in some occasion in comparison with gradient descent. I use Andrew Ng's course on Coursera but the notation really makes me a hard time to understand.

I want to know more about the derivation of the cost function $J(\theta)$ for polynomial regression and the reason why he uses the transpose of vector $x(i)$

• Andrew Ng has more than one course on Coursera, and uses subtly different notation depending on the course and topic. Please could you provide more details - the full equations you are asking about and a link to the specific lecture would be great. – Neil Slater Aug 1 '18 at 16:44
• You need to delete the questioner at SO, if you are going to ask here. Thanks. stackoverflow.com/q/51624899/7311767 – Stephen Rauch Aug 1 '18 at 18:06
• @StephenRauch is that violating something? – Media Aug 1 '18 at 18:28
• @Media, Yes it is... Cross posting is strongly discouraged... meta.stackexchange.com/q/64068/347908 – Stephen Rauch Aug 1 '18 at 18:30
• Ok I have already deleted the post in SO. It is in week 2 normal equation, the equation how to use the ϴ=(X^TX)^(-1)X^Ty or this link – Aaron Pham Aug 1 '18 at 18:58

You want to solve $$X \times \theta = Y$$. Actually, you have to find the parameters $$\theta$$. To find it, you should multiply both sides by $$X^{-1}$$ but it may have not an inverse. Consequently, you multiply each side by $$X^t$$ due to the fact that $$X^tX$$ has the inverse. After that, you multiply each side by $$(X^tX)^{-1}$$ which leads to a closed form equation for finding $$\theta$$. It should be mentioned that $$X^tX$$ does not always have an inverse. If you want to be sure it has it, you have to exploit data samples which are more than the number of features and they should not be linearly dependent otherwise the constructed matrix will not inverse and you will not be able to use it.