2
$\begingroup$

Getting into machine learning, have a reasonable background in statistics and understand the basic principles of linear algebra (matrix multiplication etc.) - but am having a damn hard time figuring out why reducing a polynomial regression works.

For example, say we have this function:

$y =$ $\beta_0$ + $\beta_1$$x_1$ + $\beta_2$$x_2$$^2$

From what I've seen on 4 videos and 6 articles, we can use the following substitutions:

  • $x_2$ = 1
  • $x_3$ = $x$
  • $x_4$ = $x^2$

To create the following model: $y =$ $\beta_0$ + $\beta_1$$x_1$ + ($\beta_2$$x_2$ + $\beta_3$$x_3$ + $\beta_4$$x_4$)

And then, fine - we can solve that as a normal multiple linear regression, and all is well.

But why, why does this work? I really cannot find an explanation for this.

Thanks!

$\endgroup$
  • $\begingroup$ I don't understand the way you index the $x$. Why is there no index in your three bullets above? $\endgroup$ – Peter Feb 15 at 11:15
1
$\begingroup$

I don't really understand your problem statement. You can simply add squared terms to any linear model. Say you have $y$ and $x$ and you want to model a polynomial function, you can write a model like:

$y=\beta_0 + \beta_1 x_1 + \beta_2 x_1^2 + u$.

In matrix form this would look like

$y=X\beta + u$.

An example with some numbers would write:

$\left( \begin{array}{rrrr} 5 \\ 8 \\ 7 \\ 6 \\ \end{array}\right) $ = $\left( \begin{array}{rrrr} 1 & 8& 8^2 \\ 1 & 4 & 4^2 \\ 1 & 3 & 3^2 \\ 1 & 6 & 6^2 \\ \end{array}\right) \beta + u.$

You can solve this like $\hat{\beta} = (X'X)^{-1} X'y$, where $\hat{\beta}$ are the coefficients of the linear regression model.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.