0
$\begingroup$

I've built a Random Forest model that classifies into four categories based on around 10 input features.

To test the accuracy, I performed 5-fold stratified cross validation using the sklearn.model_selection.StratifiedKFold method for a number of different maximum depths.

I then examined the test and train accuracy for each of these, amongst some other metrics like one-vs-one AUC. I was expecting as the maximum depth increased, the test and training accuracy would both increase, up to a point where the model starts to overfit, from which point the test accuracy would start to decrease while train accuracy continues to increase.

What I've seen instead is the test accuracy just plateauing around a certain value, even when maximum depth is increased to 100s.

I'm confused about why it's not behaving as I thought, and how to pick the optimal depth. Do I have a misunderstanding about overfitting in Random Forests?

Example data:

Max_depth  Test_acc   Train_acc

3          0.54       0.58
5          0.65       0.67
7          0.82       0.82
9          0.84       0.85
11         0.87       0.91
15         0.87       0.95
20         0.87       0.96
50         0.87       1.0
100        0.87       1.0
Improve this question
$\endgroup$
1
  • $\begingroup$ Did you check for the actual depth? Max_depth is just an upper limit. $\endgroup$
    – Broele
    Aug 17 at 22:46

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.