1
$\begingroup$

When I have the option to build a classifier like this directly

clf = RandomForestClassifier() 

why do we perform tuning by restricting the parameters like this

rf_grid = {
'bootstrap': [True, False],
 'max_depth': [10, 20],
 'min_samples_leaf': [1, 2],
 'min_samples_split': [2, 5],
 'n_estimators': [200, 400]
}

Doesn't the above tuning restrict the parameter space for RandomForestClassifier() to try.

I believe RandomForestClassifier() itself gives best model by default as it will try all possible parameters. Is it wrong? If wrong, please suggest how to find best classifier model.

$\endgroup$
0

2 Answers 2

1
$\begingroup$

FYI, as of scikit-learn 1.3.2, the RandomForestClassifier's default hyperparameters are:

  • n_estimators=100
  • criterion='gini'
  • max_depth=None
  • min_samples_leaf=1

... and so on. It does not give or try to get any best hyperparameter, just using a default.

In general, finding 'best' parameters is NP-hard, i.e. the only way is to try all possible combinations. In practice, we leverage heuristics and guesswork to limit the search space.

An exercise for you: try to train a model on a toy dataset, with and without parameter search, and observe the difference in performance.

$\endgroup$
4
$\begingroup$

You are wrong. RandomForestClassifier does not try any hyperparameters. You need to give it the specific value for each of its hyperparameters. Given such hyperparameter values, it can then train a random forest classifier.

They are called "hyperparameters" because they are not trained, they are "outside" of the specific algorithm. You need something/someone to decide what values to use.

The approach to decide which hyperparameter values to use can be anything: a person doing trial and error, a loop exploring all possible hyperparameter value combinations within some reasonable limits (i.e. grid search), a black-box optimization (e.g. skopt), etc.

Also, there is not an absolute criterion to decide which model is "best". In some cases, we might want to get the greatest accuracy, others, we might want to pay attention to the AUC, others to the F1, etc.

The variable name rf_grid in your question suggests that you have a piece of code using grid search, which loops over all possible hyperparameter configurations within some sensible limits, which in your case are defined by the rf_grid dictionary.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.