1
$\begingroup$

I'm using sklearn.model_selection.learning_curve for 5 fold training of data. The code is as given below.

train_sizes = [1, 100, 500, 1000, 2000, 3000, 3879]
train_sizes,train_scores, validation_scores = learning_curve(estimator = ensemble.RandomForestClassifier(),
                                                          X = X_res,
                                                          y = y_res, 
                                                          train_sizes = train_sizes, 
                                                          cv = 5,
                                                          scoring = 'accuracy')
train_scores_mean = train_scores.mean(axis = 1)
validation_scores_mean = validation_scores.mean(axis =1)
plt.style.use('seaborn')
plt.plot(train_sizes, train_scores_mean, label = 'Training acc')
plt.plot(train_sizes, validation_scores_mean, label = 'Validation acc')
plt.ylabel('Accuracy', fontsize = 14)
plt.xlabel('Training set size', fontsize = 14)
plt.title('Learning curves for a linear RF model', fontsize = 18, y = 1.03)
plt.legend()

Also, I'm using the random forest classifier model for a multiclass classification problem. When I plot the accuracy graph, it appears like the below. learning curve graph

How's it possible that the mean training accuracy for all training sizes is 100%?

Any help is appreciated.

$\endgroup$
1
$\begingroup$

Using all default values for the RandomForestClassifier class leads to overfitting. As stated in scikit-learn documentation,

The default values for the parameters controlling the size of the trees (e.g. max_depth, min_samples_leaf, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values.

Fully grown trees have a perfect score on train data, but as you can guess, that is not how random forests are supposed to be used. Try setting the various parameters of the classifier (start with max_depth for instance), and you will get more consistent results.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Even for a training size of 1, its 100% accuracy. How's that possible? Rather what does that mean? I didn't get it from this link scikit-learn.org/stable/modules/generated/… $\endgroup$ – Eswar Jan 8 at 7:09
  • 1
    $\begingroup$ Well, with only one sample, it is pretty easy to have a good model :) though I'm not sure how the code will perform the cross-validation in this case. $\endgroup$ – Romain Reboulleau Jan 8 at 22:24
  • $\begingroup$ I think historically, random forests were supposed to use fully-grown trees, and the bagging was supposed to be the sole reducer of overfitting. But it does seem pretty universally true in practice that limiting tree complexity gives better results. $\endgroup$ – Ben Reiniger Feb 7 at 13:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.