What regression tree ensemble methods and what parameters does Matlab actually consider in hyperparameter tuning? The documentation appears to be sparse, to say the least.

See https://se.mathworks.com/help/stats/fitrensemble.html and the example "Optimize Regression Ensemble" therein. It says "You can find hyperparameters that minimize five-fold cross-validation loss by using automatic hyperparameter optimization."

But what is the search space here?

The output in that example only displays Bag and LSBoost as methods. Does it neglect random forests, i.e. subset sampling instead of bootstrapping the input space? Or is bagging here an umbrella term that covers also random forests?

Furthermore, the output in the above example only displays NumLearnCycles (tree count), LearnRate (for boosting) and MinLeafSize (obvious). How about the CART decision tree algorithm hyperparameters? Are they included? The LSBoost algorithm appears to involve optimization of these hyperparameters (see https://se.mathworks.com/matlabcentral/answers/302103-what-is-the-algorithm-behind-lsboost-from-fitensemble-function), but, again, documentation is .. meh.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.