0
$\begingroup$

The idea was to gradually raise the degree of the polynomial. Here is the code that implements creating a random dataset, fitting the polynomial of the CHANGE_ME degree and visualising the result.

import numpy as np
import scipy.interpolate as inter
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression

np.random.seed(4)

# data generation
x_p = np.array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10])
y_p = np.array([2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 10])
poly = inter.lagrange(x_p, y_p)
def f(x):
    X = np.concatenate(
        [(x**a)[:, np.newaxis] for a in range(len(poly.coef))],
        axis = 1
    )
    return np.dot(X, poly.coef[::-1][:, np.newaxis]).ravel()

y = lambda x: f(x) + np.random.normal(0, 1, x.shape)

X_sample = np.sort(np.random.uniform(0, 10, 200))
Y_sample = y(X_sample)


# models
def get_poly_matrix(X, p = 2):
    return np.concatenate(
        [np.array(X)[:, np.newaxis]**(i) for i in range(p+1)],
        axis = 1
    )
def get_poly_predict(X, y, p = 2):
    
    X_matr = get_poly_matrix(X, p)
    return LinearRegression(
        fit_intercept=False
    ).fit(X_matr, y).predict(X_matr)


CHANGE_ME = 20
plt.scatter(X_sample, Y_sample, color = "black")
plt.title("polynomial degree " + str(CHANGE_ME))
plt.plot(
    X_sample,
    get_poly_predict(X_sample, Y_sample, CHANGE_ME),
    linewidth = 4
)

As long as the degree of the polynomial has not reached 15 all is well - the model is fitting closer and closer to the training data.

enter image description here

After that, predictions for small X become zero, like:

enter image description here

And with the further increase in the degree, it only gets worse. It seems to be a matter of float precision. But anyway, I don't know how to fix it.

$\endgroup$
1
  • $\begingroup$ There is almost never a good reason to use a polynomial beyond degree 3, so you may want to rethink your approach. $\endgroup$ Feb 3, 2023 at 13:38

1 Answer 1

2
$\begingroup$

Try fit_intercept=True

The fit intercept option determines whether the model should fit an intercept.

fit_intercept=True

fit)intercept=False

$\endgroup$
3
  • $\begingroup$ A intercept has already been included in the model, by including a column of ones in the training data. But the approach you suggested had an impact on the predictions. The model still fits the data poorly when the degree is increased, now the predictions are not near zero but ~8 which corresponds to the mean value of y. $\endgroup$
    – Dranikf
    Feb 3, 2023 at 14:25
  • $\begingroup$ Such outcomes are expected since the model has gotten too complex for the data. If your goal isn't only to see the difference, try a larger and more complicated sample. $\endgroup$ Feb 3, 2023 at 14:44
  • $\begingroup$ this may give you better results: X_sample = np.sort(np.random.uniform(0, 5, 20000)) $\endgroup$ Feb 3, 2023 at 14:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.