1
$\begingroup$

Would be grateful if some expert on the forum can help me understand how to decide optimum number of leaves in a decision tree analysis.

I am using SAS and if I supply leaves=6 in my model then miss-classification rates for validation & training data sets are 18.6% & 18.8% respectively. And SAS lists 5 variables which are significant.

And if I don't supply leaves count in the code and let SAS decide it, then SAS after pruning takes 10 as leaves count and miss-classification rates for validation & training data sets are 17.5% & 16.9% respectively. And SAS lists 6 variables which are significant.

Now that the miss-classification rates have reduced & trees after pruning have increased from 4 to 10, is it a good thing or it indicates overfitting?

Looking forward to opinions of experts in this group. Thanks

$\endgroup$
1
$\begingroup$

I'll assume that your test and validation datasets have been created appropriately (e.g. no observations are in both test and validation sets, both sets are of appropriate size, etc.).

Overfitting means that your model fits very well to your training data but does not generalise well on unseen data (i.e. will perform poorly on your validation dataset).

Your misclassification rate on the validation set (unseen data) is decreasing, and is therefore a good thing. However, if the misclassification rate on the validation set were to increase, that would indicate overfitting.

$\endgroup$
2
  • $\begingroup$ Thanks Brad, I understand now. Really appreciate your help. Have a nice day. $\endgroup$ Mar 28 '19 at 10:47
  • $\begingroup$ No problem. If you're happy with the solution, then please mark as the answer so the question can be closed. $\endgroup$
    – bradS
    Mar 28 '19 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.