1
$\begingroup$

Consider I have one dependent variable to predict 'Attitude' which can take three values 'Positive/Negative/Neutral'. I have following independent variables or features- Age, Height, Gender, Income etc. I trying to predict Attitude using decision tree classifier.

Attitude ~ Age + Height + Gender + Income (Decision Tree)

I am getting >90% accuracy for the when tree depth is 15. As tree is dividing on continuous variables (i.e. Age, Income and Height) again and again to get leaf with pure classes.

Is this problem of overfitting? Should I convert the continuous variables into categorical variables (like range classes)?

$\endgroup$
  • $\begingroup$ You could look at out of bag testing. To see the performance of trees where data points were not used in the construction of that specific tree. $\endgroup$ – Harpal Jul 21 '16 at 20:34
3
$\begingroup$

There is no need to split continuous variables because the tree already does that automatically. The only way you can test for overfitting is by either using a holdout set or by doing cross validation. If you are overfitting, changing a continuous variable to a categorical variable likely won't make a difference. If you get the sense that you're overfitting, you should reduce the depth of your tree.

$\endgroup$
  • 2
    $\begingroup$ To add to what Ryan said, you might actually take away the natural freedom that the decision tree algorithm has in determining splits by converting to a categorical. E.g. let's say the best spilt for your problem is at 25 for age. If you pre- split and create a 20-30 bucket you will take away the option of splitting at 25. $\endgroup$ – wabbit Jul 11 '16 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.