I am doing hyperparameter tuning + cross validation and I'm constantly getting that the optimal size of the leaf should be 1. Should I worry? Is this a sign of overfitting?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
leaf_size should have zero effect on the performance of the model. Its effect is on construction of the lookup object, which affects training and prediction time, but not the results.
The best parameters are probably just breaking ties by value of this parameter, so the smallest is chosen. You can inspect the GridSearchCV attribute cv_results_ to be sure. It should also contain training and prediction times, which might inform what value you ultimately want to select.
answered May 3, 2022 at 17:51