Making sense of a accuracy plot for a 5 fold training using random forest

Question

I'm using sklearn.model_selection.learning_curve for 5 fold training of data. The code is as given below.

train_sizes = [1, 100, 500, 1000, 2000, 3000, 3879]
train_sizes,train_scores, validation_scores = learning_curve(estimator = ensemble.RandomForestClassifier(),
                                                          X = X_res,
                                                          y = y_res, 
                                                          train_sizes = train_sizes, 
                                                          cv = 5,
                                                          scoring = 'accuracy')
train_scores_mean = train_scores.mean(axis = 1)
validation_scores_mean = validation_scores.mean(axis =1)
plt.style.use('seaborn')
plt.plot(train_sizes, train_scores_mean, label = 'Training acc')
plt.plot(train_sizes, validation_scores_mean, label = 'Validation acc')
plt.ylabel('Accuracy', fontsize = 14)
plt.xlabel('Training set size', fontsize = 14)
plt.title('Learning curves for a linear RF model', fontsize = 18, y = 1.03)
plt.legend()

Also, I'm using the random forest classifier model for a multiclass classification problem. When I plot the accuracy graph, it appears like the below.

How's it possible that the mean training accuracy for all training sizes is 100%?

Any help is appreciated.

Answer 1

Using all default values for the RandomForestClassifier class leads to overfitting. As stated in scikit-learn documentation,

The default values for the parameters controlling the size of the trees (e.g. max_depth, min_samples_leaf, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values.

Fully grown trees have a perfect score on train data, but as you can guess, that is not how random forests are supposed to be used. Try setting the various parameters of the classifier (start with max_depth for instance), and you will get more consistent results.