Different values of mean absolute error when using GridSearchCV for max_leaf_nodes vs manually optimising max_leaf_nodes

I am trying out hyperparameter tuning vs manually selecting the best parameter (max_leaf_nodes) on a decision tree model with mean absolute error as the scoring. In theory, both should give me the same MAE and max_leaf_nodes; but, both are giving me different MAEs. Also, if I change the value of cv in GridSearchCV I get different results. So basically I have two questions:

1. Why am I getting different max_leaf_nodes and MAE in both cases?

2. How do I determine the value of cv in GridsearchCV, because I get different results for cv = 3, cv = 5, and cv = 10?

• I am not sure how big the difference is, but since you are not specifying a seed in the second example the results will differ depending on the seed. Jul 1 '21 at 17:28
• I tried setting the seed to 69 everywhere. But still I'm getting different values for mean absolute error and max_leaf_nodes. When performing GridSearchCV mae= 27378 and best max_leaf_nodes = 100 but whne manually optimizing mae = 28202 and max_leaf_nodes = 50! Jul 1 '21 at 18:03
• That might be explained by the default scorer that is used when using GridSearchCV, see also this stackexchange answer. Jul 1 '21 at 18:17
• Please do not insert images of code; paste the code into a code-formatted block instead. Jul 1 '21 at 18:31
• @Oxbowerce I've used the same scoring (mean absolute error) in both the cases. Also the seeds are same everywhere. Still not getting similar results! Jul 2 '21 at 5:42

Your manual approach gives the MAE on the test set. Because you've set an integer for the parameter cv, the GridSearchCV is doing k-fold cross-validation (see the parameter description in grid search docs), and so the score .best_score_ is the average MAE on the multiple test folds.
If you really want a single train/test split, you can do that in GridSearchCV, see e.g. this SO post.