5
$\begingroup$

As I understand it one can use cross validation to help find the optimal pruning of a classification or regression tree, for example, implemented here in Matlab.

I do not understand the following about this process. Since each cross validation fold has random data, it may be that the trees fit on each data set may be different and have differing depths. How can one find the best pruning level over all such trees then?

$\endgroup$
2
  • 2
    $\begingroup$ I think that the answer wants to say: take the pruning depth as another parameter to tweak, i.e. take a tree algorithm with a fixed set of parameters (i.e. depth is fixed) and then let it run over different data sets and compute some function (usually the mean) of the performances over all the data sets and assign this as the total performance of this parameter setting. $\endgroup$ Oct 26 '15 at 9:06
  • $\begingroup$ Take a look at ID3 algorithm used to generate a decision tree from a database en.wikipedia.org/wiki/ID3_algorithm $\endgroup$
    – sonaam1234
    Aug 30 '16 at 8:38
1
$\begingroup$

I don't know how this is implemented in matlab. I know that some packages use cross validation to decide whether to grow the tree or not. Quite simply, they decide whether to grow based on that evaluation.

(Notice what a lot of packages call pruning is: during training, they mark branches with some score criterium and then remove if the user wants afterwards.)

Personally, I use sklearn which does not have this feature. So, I just do a grid search for several values of max_depth and use whatever maximizes accuracy or whatever score I want.

$\endgroup$
0
$\begingroup$

Develop a training set, validation set and test set. Training set contains 40% of observations, validation and test set contain 30% each of the observations (assuming you have a fairly large dataset in the first place). Develop 5 decision trees, each with differing parameters that you would like to test. Run these decision trees on the training set and then validation set and see which decision tree has the lowest ASE (Average Squared Error) on the validation set. You can use a different validation criterion if you so choose but I prefer the ASE. If you feel that the best performing tree can still perform better given the ASE on the validation set, then go back and tweak the parameters of this tree and re-run on training and validation set. Once you are happy, use your final tree and predict on your test set.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.