1
$\begingroup$

Consider I have one dependent variable to predict 'Attitude' which can take three values 'Positive/Negative/Neutral'. I have following independent variables or features- Age, Height, Gender, Income etc. I trying to predict Attitude using decision tree classifier.

Attitude ~ Age + Height + Gender + Income (Decision Tree)

I am getting >90% accuracy for the when tree depth is 15. As tree is dividing on continuous variables (i.e. Age, Income and Height) again and again to get leaf with pure classes.

Is this problem of overfitting? Should I convert the continuous variables into categorical variables (like range classes)?

Improve this question
$\endgroup$
1
  • $\begingroup$ You could look at out of bag testing. To see the performance of trees where data points were not used in the construction of that specific tree. $\endgroup$
    – Harpal
    Jul 21, 2016 at 20:34

1 Answer 1

Reset to default
3
$\begingroup$

There is no need to split continuous variables because the tree already does that automatically. The only way you can test for overfitting is by either using a holdout set or by doing cross validation. If you are overfitting, changing a continuous variable to a categorical variable likely won't make a difference. If you get the sense that you're overfitting, you should reduce the depth of your tree.

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ To add to what Ryan said, you might actually take away the natural freedom that the decision tree algorithm has in determining splits by converting to a categorical. E.g. let's say the best spilt for your problem is at 25 for age. If you pre- split and create a 20-30 bucket you will take away the option of splitting at 25. $\endgroup$
    – wabbit
    Jul 11, 2016 at 14:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.