1
$\begingroup$

The description on the RandomizedSearchCV says this about best hyperparameters :

"Estimator that was chosen by the search, i.e. estimator which gave highest score (or smallest loss if specified) on the left out data. Not available if refit=False."

Lets say i used RandomizedSearchCV with 5 folds. Lets say fold number 2 had best validation score out of five, then the hyperparameters of the training data of that model would be returned as best hyperparameters?

I thought the hyperparameters would be same for each fold in a given iteration and hence the iteration with highest validation score will be the best hyperparameter.

Are hyperparameters in each fold for one iteration different for some methods ?

$\endgroup$
1
  • 1
    $\begingroup$ You are right. In CV, the validation score is computed for each of the k-folds and then averaged. This is done for every (sampled in Randomized CV) combination of hyperparameters. The combination that yields the best average score is returned. $\endgroup$ Jun 14, 2023 at 1:01

2 Answers 2

1
$\begingroup$

Sometimes these values are not used anymore, they are just used to calculate the best model for each fold and its prediction error (on the validation folds). And if the error satisfies you, at each fold or on average then the model is fitted again (both the parameters and the hyperparameters) with all the (test+validation) data and tested again with the test data.

Sometimes it's used taking the average of the hyperparameters on all partitions, or even better we can create an ensemble model averaging all models.

In fact the hyperparameters are not important in the final model, they are used just during the fitting process.

$\endgroup$
0
$\begingroup$

For each fold, the HP will be different. Hence, the fold which gives the maximum accuracy or minimum error, the HP of that particular fold will be returned.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.