Using GridSearchCV and a Random Forest Regressor with the same parameters gives different results

As the huge title says I'm trying to use GridSearchCV to find the best parameters for a Random Forest Regressor and I'm measuring my results with mse.

Inputs_Treino = dataset.iloc[:253,1:4].values
Outputs_Treino = dataset.iloc[:253,-1].values
Inputs_Teste = dataset.iloc[254:,1:4].values
Outputs_Teste = dataset.iloc[254:,-1].values

estimator = RandomForestRegressor()
para_grids = {
"n_estimators" : [10,50,100],
"max_features" : ["auto", "log2", "sqrt"],
"bootstrap"    : [True, False]
}

grid = GridSearchCV(estimator, para_grids, scoring = 'mean_squared_error')
grid.fit(Inputs_Treino, Outputs_Treino)
forest = grid.best_estimator_

reg_prediction=forest.predict(Inputs_Teste)

print (grid.best_score_, grid.best_params_)

mse = mean_absolute_error(Outputs_Teste, reg_prediction)


This is the gist of the code (nothing too complex I know, just getting started with it all)

When I print the result of grid.best_estimator_ I get this

RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,
max_features='auto', max_leaf_nodes=None,
min_impurity_split=1e-07, min_samples_leaf=1,
min_samples_split=2, min_weight_fraction_leaf=0.0,
n_estimators=10, n_jobs=1, oob_score=False, random_state=None,
verbose=0, warm_start=False)


The problem is if I try to create a regressor with these parameters (without using grid search at all) and train it the same way I get a waaaay bigger MSE on the testing set (5.483837301587303 vs 43.801520165079467)

Inputs_Treino = dataset.iloc[:253,1:4].values
Outputs_Treino = dataset.iloc[:253,-1].values
Inputs_Teste = dataset.iloc[254:,1:4].values
Outputs_Teste = dataset.iloc[254:,-1].values

regressor = RandomForestRegressor(bootstrap=True, criterion='mse', max_depth=None,
max_features='auto', max_leaf_nodes=None,
min_impurity_split=1e-07, min_samples_leaf=1,
min_samples_split=2, min_weight_fraction_leaf=0.0,
n_estimators=10, n_jobs=1, oob_score=False, random_state=None,
verbose=0, warm_start=False)

regressor.fit(Inputs_Treino,Outputs_Treino)

#fazer as predictions
Teste_Prediction = regressor.predict(Inputs_Teste);

mse = mean_squared_error(Outputs_Teste, Teste_Prediction);


Does this have to do with the cross validation GridSearchCV performs ? What am I missing here ?

• n_estimators=10, VS n_estimators=100, - maybe not the reason, but... – f.g. Oct 15 '18 at 21:16
• @f.g. oh that's a mistake on my part but the result is the same. – Gabriel Silva Oct 15 '18 at 21:31
• Also, mean_absolute_error vs mean_squared_error. That seems reasonably likely to explain at least a large part of the difference. – Ben Reiniger Apr 15 '19 at 19:23

To reproduce results across runs you should set the random_state parameter. For example:
estimator = RandomForestRegressor(random_state=420)