4
$\begingroup$

As I understand, ExtraTreesRegressor from sklearn works by doing random splits instead of minimizing a metric like gini for classification or mae for regression.

I don't understand why there's a criterion parameter, as the criterion for the splits should be random.

Is it just for code compatibility, or am I missing something?

Improve this question
$\endgroup$
4
$\begingroup$

No, extremely-random trees does still optimize splits. It does only pick one random splitting point for each feature (out of those randomly chosen max_features) but then which feature is actually used for the split depends on the criterion chosen.

https://scikit-learn.org/stable/modules/ensemble.html#extremely-randomized-trees

Improve this answer
$\endgroup$
  • $\begingroup$ You're right, the link says it clearly "As in random forests, a random subset of candidate features is used, but instead of looking for the most discriminative thresholds, thresholds are drawn at random for each candidate feature and the best of these randomly-generated thresholds is picked as the splitting rule." $\endgroup$ – David Masip Jun 17 '20 at 13:24
1
$\begingroup$

The criterion parameter is used to measure the quality of the split when selected, it is not involved in the initial splitting algorithm (the features used for the split are chosen randomly)

ExtraTreesRegressor:

  • mse and mae are the only options available for use, and mse is the default. mae was added after version 0.18. Check your version if it is available. A few issues have been reported with the use of mae.

Reference:

https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.ExtraTreesRegressor.html

criterion{“mse”, “mae”}, default=”mse” The function to measure the quality of a split. Supported criteria are “mse” for the mean squared error, which is equal to variance reduction as feature selection criterion, and “mae” for the mean absolute error.

Improve this answer
$\endgroup$
  • 1
    $\begingroup$ Once measured the quality of the split, what do you do? Is not useful anywhere, right? $\endgroup$ – Carlos Mougan Jun 17 '20 at 11:34
  • 1
    $\begingroup$ Yeah, then you're telling that, with the same seed, training two models, one with criterion mse and the other with criterion mae should give the same predictions? $\endgroup$ – David Masip Jun 17 '20 at 11:37
  • $\begingroup$ @David, maybe i misunderstood your statement about "criterion for the splits should be random". The features for the splits are selected randomly. But which split is chosen will be based on the optimization using the criterion selected. $\endgroup$ – Donald S Jun 17 '20 at 13:46
  • $\begingroup$ yeah I think I misunderstood as well $\endgroup$ – David Masip Jun 17 '20 at 14:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.