In machine learning, an overfitted model fits training set very well but cannot generalize to new instances. I evaluated my model using cross-validation and my accuracy drops when setting the maximum number of splits of my decision tree beyond a certain number. Can this be associated with overfitting?
UPDATE
Here is the result. I am doing Human Activity Recognition in MATLAB with 16 features.
It definitely can be associated with over-fitting. I would suggest plotting the training and validation accuracy in a single graph. So you see how both depend on the number of splits. If it is over-fitting one would expect that the training accuracy continues rising, while the validation accuracy gets to a maximum and then drops.
However, you have to make sure that your training and validation sets are OK. I assume you do something like k-fold cross validation. Where every sample is k-times in a training set and one time in a validation set. And you have enough samples to train your model etc.
In decision trees, one impact of increasing the maximum number of splits is that it can lead to overfitting on the training data set because effectively we are increasing the flexibility of the algorithm to memorize the train set. A decrease in the CV accuracy is understandable in this case.
Since it is a common issue with any ml parameter, the go-to option is to search over a set of options and pick the one that gives the best CV score. On the other hand for Decision trees there are more sophisticated methods that fall largely either in the "top-down" vs "bottom-up" category.
A top down approach would be to keep increasing the max num of splits (holding everything else constant) until you see a drop in CV score.
A Bottom up approach is to grow a full tree first, and then prune/coalesce/average leaves to reduce the variance of the predictions.
one reference for these two options(page 9): [http://www.ccs.neu.edu/home/vip/teach/MLcourse/1_intro_DT_RULES_REG/lecture_notes/lectureNotes_DecisionTree_Spring15.pdf][1]
