# high degree polynomial model with sklearn does not fit

The idea was to gradually raise the degree of the polynomial. Here is the code that implements creating a random dataset, fitting the polynomial of the CHANGE_ME degree and visualising the result.

import numpy as np
import scipy.interpolate as inter
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

np.random.seed(4)

# data generation
x_p = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y_p = np.array([2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 10])
poly = inter.lagrange(x_p, y_p)
def f(x):
X = np.concatenate(
[(x**a)[:, np.newaxis] for a in range(len(poly.coef))],
axis = 1
)
return np.dot(X, poly.coef[::-1][:, np.newaxis]).ravel()

y = lambda x: f(x) + np.random.normal(0, 1, x.shape)

X_sample = np.sort(np.random.uniform(0, 10, 200))
Y_sample = y(X_sample)

# models
def get_poly_matrix(X, p = 2):
return np.concatenate(
[np.array(X)[:, np.newaxis]**(i) for i in range(p+1)],
axis = 1
)
def get_poly_predict(X, y, p = 2):

X_matr = get_poly_matrix(X, p)
return LinearRegression(
fit_intercept=False
).fit(X_matr, y).predict(X_matr)

CHANGE_ME = 20
plt.scatter(X_sample, Y_sample, color = "black")
plt.title("polynomial degree " + str(CHANGE_ME))
plt.plot(
X_sample,
get_poly_predict(X_sample, Y_sample, CHANGE_ME),
linewidth = 4
)

As long as the degree of the polynomial has not reached 15 all is well - the model is fitting closer and closer to the training data.

After that, predictions for small X become zero, like:

And with the further increase in the degree, it only gets worse. It seems to be a matter of float precision. But anyway, I don't know how to fix it.

• There is almost never a good reason to use a polynomial beyond degree 3, so you may want to rethink your approach. Feb 3, 2023 at 13:38