1
$\begingroup$

I want to ask you a question. Suppose I use the following RandomizedSearchCV to find the model's best hyperparams:

classification = RandomizedSearchCV(XGBClassifier(n_jobs=-1, random_state=42), {
    'max_depth': [1,2,3,4,5,6,7,8,9,10],
    'learning_rate': [.001, .005, .01, .05, .1, .2, .3],
    'n_estimators': [300, 500, 1000, 1500],
    'min_child_weight': [1,2,3,4,5,6,7,8,9,10],
    'subsample': [.4, .5, .6, .7, .8, .9, 1],
    'colsample_bytree': [.4, .5, .6, .7, .75, .77, .79 ,.8, .85],
    'gamma': [0.0, 0.1, 0.2, 0.3, 0.4, 0.5],
    'reg_lambda':[0, 0.5, 1, 1.5, 2, 3, 4.5]
}, cv=5, return_train_score=False, scoring='roc_auc', n_iter=5)

classification.fit(X,y)

Then I print the best parameters using .best_params_

Later can I train a separate model using those parameters like this:

model = XGBClassifier(n_jobs=-1, n_estimators=500, max_depth=2, subsample=0.8, gamma=0, colsample_bytree=0.5, learning_rate=0.2,
                      reg_lambda=4.5, min_child_weight=2 ,random_state=42)
model.fit(X,y) 

I have used X and y (the training set) to train both the RandomizedSearchCV and the model, in order not to repeat every time the params' search.

Is this correct?

$\endgroup$

1 Answer 1

2
$\begingroup$

You can but this is not absolutely necessary:

the model with parameters best_params_ is stored in best_estimator_ as long as you set refit=True when instantiating RandomizedSearchCV.

check the doc

You might want to refit yourself if you want to use the full training set after using cross-validation.

$\endgroup$
2
  • $\begingroup$ What if refit=False? How am I supposed to use grid.best_params_ to instantiate a model and re-fit on training data? $\endgroup$
    – tail
    Sep 8, 2023 at 10:01
  • $\begingroup$ You just instantiate a new XGBClassifier with these parameters. $\endgroup$ Sep 9, 2023 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.