1
$\begingroup$

I'm trying to do hyperparameter tunning for random forest regression model. My question is- is there any order I should do it? like starting with specific parameter and then move on on the other? should I check the model each time with one paramter only or for each parameter i'm checking I should add the ones I have found already?

Exmaple: if I run model with only n_job -1 and min_sample_leaf, and I got the best value for min_sample_leaf and now I want to check the max depth, and after that more parameters, should I put already the best value I found for min)sample_leaf? or I should check it without any other parameter and in the end create model with the best values from each one?

$\endgroup$

2 Answers 2

1
$\begingroup$

Have a look at this blog post: https://towardsdatascience.com/hyperparameter-tuning-the-random-forest-in-python-using-scikit-learn-28d2aa77dd74

Ideally you should optimise the hyperparameters jointly and not one after the other. More importantly, you should be doing cross validation. Consider also the RandomizedSearchCV described in the post.

$\endgroup$
1
$\begingroup$

Since the various hyperparameter are related you cannot be sure that - for instance - a tuned value for min_samples_split from a model where all other hyperparameter are set to default values, will be generally optimal. When you have a situation where you limit the depth of trees (by max_depth assuming you use sklearn), min_samples_split may be a non-binding constraint. It may however be relevant when you set max_depth=None, since as stated in the docs:

If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

From a practical perspective I would start tuning n_estimators (more tend to be better), max_depth, and max_features (square root of the number of features is a good starting point) with a fixed random_state. Contingent on the reults you may start adding additional constraints.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.