I am using RandomForestRegressor
(scikit-learn
python package). I am looking for the best values for hyperparameters n_estimators
and min_samples_split
for fitting my regressor on a train dataset ( X_train
, y_train
) :
param_grid = { 'n_estimators' : range( 10 , 201 ) ,
'min_samples_split' : range( 2 , 11 ) }
rfr = RandomForestRegressor()
best_score = 0
best_param = {}
for param in ParameterGrid( param_grid ):
rfr.set_params( **g )
rfr.fit( X_train , y_train )
score = rfr.score( X_train , y_train )
if score > best_score :
best_score = score
best_param = param
print( 'best_score : {0}'.format( best_score ) )
print( 'best_parameters : {0}'.format( best_param ) )
I am not using cross validation on purpose.
The best hyper parameters are in the best_param
variable , the corresponding score in the best_score
variable.
After that, I set the hyper parameters of my regressor using the best ones :
rfr.set_params( **best_param )
I fit the regressor on my train dataset:
rfr.fit( X_train , y_train )
Finally just to check I obtain the score of the regressor with the best hyper parameters:
rfr.score( X_train , y_train )
The score is not the same as the one calculated in the grid : 0.806 out of grid versus 0.963 in the grid.
I do not understand why. The hyper parameters are the best ones, and the dataset used to calculate these best parameters is the same. Any hints?