I don't quite seem to understand the rules used to create the polynomial features when trying to find a polynomial model with Linear Regression in the multivariate setting.

Let's say I have a two predictor variables a and b. When generating polynomial features (for example using sklearn) I get 6 features for degree 2: y = bias + a + b + a * b + a^2 + b^2

This much I understand. When I set the degree to 3 I get 10 features instead of my expected 8. I expected it to be this: y = bias + a + b + a * b + a^2 + b^2 + a^3 + b^3

What is the general formula of generating multivariate features? How does this look like in the 3rd degree?


General Formula is as follow: \begin{equation} N(n,d)=C(n+d,d) \end{equation} where n is the number of the features, $d$ is the degree of the polynomial, $C$ is binomial coefficient(combination).

Example with vector (2,3) to 3rd degree :

x = np.array[[2,3]]
pf = PolynomialFeatures(degree=3, include_bias=True)

returns :

array([[ 1.,  2.,  3.,  4.,  6.,  9.,  8., 12., 18., 27.]])

wich is : [1, $\:$ $x_1$, $\:$ $x_2$, $\:$ $x_1^2$,$\:$ $x_1x_2$, $\:$ $x_2^2$, $\:$ $x_1^3$, $\:$ $x_2x_1^2$, $\:$ $x_2^2 x_1$, $\:$ $x_2^3$]

You can see this as expanding this equation and throwing out the coefficients :

\begin{equation}1 + (x_1 + x_2) + (x_1 + x_2)^2 + (x_1 + x_2)^3 \end{equation}


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.