# Validation Curve Interpretations for Decision Tree

I'm working on a machine learning class, and we're using supervised learning right now, starting with decision trees. I'm using the UCI Credit Card dataset (whether or not certain people will default in their payments due to past history).

Using a decision tree classifier for this attempt. Running a validation curve using scikit-learn, I'm getting a plot I'm not quite sure how to interpret. As you can see from the axes, the parameter is min_samples_leaf, and I'm varying it from 1 to 30 (by 2).

Based on this plot and some Googling, I believe the correct way to interpret this is that this dataset has high bias with no variance and nothing is really being learned. Or, in other words, decision trees are not a good algorithm for this dataset, since there doesn't really seem to be a trade-off.

For max depth, I'm getting a validation curve that looks like this:

Based on what I see here, there is quite a bit of bias at the smaller set, and more variance as the depth increases. Given that GridSearchCV returns an ideal max_depth of 5 and min_samples_leaf of 19. (edit: corrected numbers). Those numbers seem to indicate a very high bias, and there really is nothing to be learned here using decision trees.

Overall, based on the min_samples_leaf, I would hesitate to recommend a decision tree for this data set. However, the learning curve and the max_depth validation curve both seem to show there might be some value.

Puzzling to me is that the accuracy score (using metrics.accuracy_score and the ideal parameters from GridSearchCV) is 82%, which doesn't seem that bad. How do I reconcile these crappy validation curves with the accuracy score?

• both parameters min_samples_leaf and max_depth are not very important for decision trees, so it's not surprising not to see much variation (or not all) across different values:
• the fact that min_sample_leaf doesn't influence the performance simply means that the algorithm finds enough good predictors in the features to create leaves with a high number of instances, apparently always more than 30.
• I'm less sure about max_depth, but I assume that it's a parameter which is used to prune the tree in order to avoid overfitting. This is exactly what happens here as soon as it's increased above 5: the algorithm creates a deeper (more complex) tree by using very specific conditions on the features which turn out to be specific to the training set, hence the divergence between training score and CV score (that's a clear sign of overfitting).
• Value 5 for max_depth is indeed optimal, as seen on the second graph (due to overfitting with higher values). The value 19 returned by grid search for min_samples_leaf is as optimal as any other value between 1 and 30, as seen on the first graph: grid search just happened to pick this one but it doesn't have any impact anyway.