How can I improve my predictive model?

Here is my interpretation of my model so far, I am investigating the relationship between ratings and followers on video games, but there is a problem. The more you get high ratings, the more you get followers, but very few of them.

from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import cross_validate, learning_curve

polynomial_features = PolynomialFeatures(degree=2, include_bias=False)
X_poly = polynomial_features.fit_transform(X)
model_reg = LinearRegression()
cv_result = cross_validate(model_reg, X_poly, y, cv = 5)
cv_result['test_score'].mean()

#0.031169232070832886


The $$R^2$$ is very low, but that should not be a surprise given the prediction plot below.

model = LinearRegression()
sorted_df = games_top.sort_values('rating')

sorted_X = sorted_df[['rating']]
sorted_y = sorted_df['followers']

# Creates polynomial
poly_features = PolynomialFeatures(degree=2, include_bias=False)
X_poly_sorted = poly_features.fit_transform(sorted_X)

model.fit(X_poly_sorted, sorted_y)
predictions = model.predict(X_poly_sorted)
#plot predictions over original data
%matplotlib widget
sns.scatterplot(x=sorted_X['rating'], y=sorted_y, alpha=0.5)
plt.plot(sorted_X['rating'], predictions, linewidth=3, color='r')


The first thing I can notice is that your followers feature ranges from $$0$$ to $$10^6$$. You would want to first make your regression on a logarithmic scale:

import numpy as np
sorted_log_y = np.log(sorted_y)

poly_features = PolynomialFeatures(degree=2, include_bias=False)
X_poly_sorted = poly_features.fit_transform(sorted_X)

model.fit(X_poly_sorted, sorted_log_y)
predictions = model.predict(X_poly_sorted)



Also, are you sure you dependant variable has a polynomial relation with the regressors ? I would suggest investigate first without PolynomialFeatures transformation.

And finally, if the conditions for OLS to be the BLUE OLS hypothesis are not respected, a regularization approach could help you improve your score:

from sklearn.linear_model import LassoCV

reg = LassoCV(cv=5, random_state=0).fit(X_poly_sorted, sorted_log_y)
reg.score(X_poly_sorted, sorted_log_y)



Check the theory behind Lasso here.

• I have to precise that if your followers features takes 0 values. You have to take $log(1 + sorted_y)$ as the dependent variable instead.
– Yann
Commented May 17 at 9:04