Ridge regression model creation using grid-search and cross validation

Question

I created python code for ridge regression.For that I used cross validation and grid-search technique in together. i got output result. I want check whether my regression model building steps correct or not? can some one explain it?

from sklearn.linear_model import Ridge
ridge_reg = Ridge()
from sklearn.model_selection import GridSearchCV
params_Ridge = {'alpha': [1,0.1,0.01,0.001,0.0001,0] , "fit_intercept": [True, False], "solver": ['svd', 'cholesky', 'lsqr', 'sparse_cg', 'sag', 'saga']}
Ridge_GS = GridSearchCV(ridge_reg, param_grid=params_Ridge, n_jobs=-1)
Ridge_GS.fit(x_train,y_train)
Ridge_GS.best_params_

output - {'alpha': 1, 'fit_intercept': True, 'solver': 'cholesky'}

Ridgeregression = Ridge(random_state=3, **Ridge_GS.best_params_)
from sklearn.model_selection import cross_val_score
all_accuracies = cross_val_score(estimator=Ridgeregression, X=x_train, y=y_train, cv=5)
all_accuracies

output - array([0.93335508, 0.8984485 , 0.91529146, 0.89309012, 0.90829416])

print(all_accuracies.mean())

output - 0.909695864130532

Ridgeregression.fit(x_train,y_train)
Ridgeregression.score(x_test,y_test)

output - 0.9113458623386644

Is 0.9113458623386644 my ridge regression accuracy(R squred) ? if it is, then what is meaning of 0.909695864130532 value.

Answer 1

Is 0.9113458623386644 my ridge regression accuracy(R squred) ? if it is, then what is meaning of 0.909695864130532 value.

These are both R^2 values.

The first score is the cross-validation score on the training set, and the second is your test set score. The first is perhaps a little biased, since those models are built using hyperparameters selected while using that dataset (but a different cv-split I think). The second score should be unbiased, and should probably be your reported test score. (The second score is a bit better than what should be a optimistically-biased score; perhaps just randomly, perhaps because of the train/test split, and perhaps because that model has seen more data.)

I want check whether my regression model building steps correct or not? can some one explain it?

Things look OK to me.

You could save yourself some code and training time; by default GridSearchCV refits a model on the entire training set using the identified hyperparameters, so you don't need to fit in the last code block. It also has the cv_results_ and best_score_ attributes to provide you with cross-validation scores, but since you've used the them in selecting the optimal hyperparameters, the best score is no longer an unbiased estimator for future performance.

Note too that there's a builtin for tuning the regularization parameter, but it uses a different CV approach and has RMSE as default scorer.