# high degree polynomial model with sklearn does not fit

The idea was to gradually raise the degree of the polynomial. Here is the code that implements creating a random dataset, fitting the polynomial of the CHANGE_ME degree and visualising the result.

import numpy as np
import scipy.interpolate as inter
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

np.random.seed(4)

# data generation
x_p = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y_p = np.array([2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 10])
poly = inter.lagrange(x_p, y_p)
def f(x):
X = np.concatenate(
[(x**a)[:, np.newaxis] for a in range(len(poly.coef))],
axis = 1
)
return np.dot(X, poly.coef[::-1][:, np.newaxis]).ravel()

y = lambda x: f(x) + np.random.normal(0, 1, x.shape)

X_sample = np.sort(np.random.uniform(0, 10, 200))
Y_sample = y(X_sample)

# models
def get_poly_matrix(X, p = 2):
return np.concatenate(
[np.array(X)[:, np.newaxis]**(i) for i in range(p+1)],
axis = 1
)
def get_poly_predict(X, y, p = 2):

X_matr = get_poly_matrix(X, p)
return LinearRegression(
fit_intercept=False
).fit(X_matr, y).predict(X_matr)

CHANGE_ME = 20
plt.scatter(X_sample, Y_sample, color = "black")
plt.title("polynomial degree " + str(CHANGE_ME))
plt.plot(
X_sample,
get_poly_predict(X_sample, Y_sample, CHANGE_ME),
linewidth = 4
)


As long as the degree of the polynomial has not reached 15 all is well - the model is fitting closer and closer to the training data.

After that, predictions for small X become zero, like:

And with the further increase in the degree, it only gets worse. It seems to be a matter of float precision. But anyway, I don't know how to fix it.

• There is almost never a good reason to use a polynomial beyond degree 3, so you may want to rethink your approach. Commented Feb 3, 2023 at 13:38

Try fit_intercept=True
• A intercept has already been included in the model, by including a column of ones in the training data. But the approach you suggested had an impact on the predictions. The model still fits the data poorly when the degree is increased, now the predictions are not near zero but ~8 which corresponds to the mean value of y. Commented Feb 3, 2023 at 14:25