# Evaluating the performance of a random forest classifier

I'm using a random forest classifier (in R) to impute missing data in a dataset. Basically, I have a bunch of objects (companies) and I want to guess an attribute (size) from other attributes (capital, owning_group and state). The dependent attribute is a categorical variable (size) with 3 possible values (small|medium|large). A random forest (R package randomForest) on a set of 3 variables provide this output:

ff = size ~ capital + owning_group + state

Call:
randomForest(formula = ff, data = df, importance = T, ntree = ntree, na.action = na.omit)
Type of random forest: classification
Number of trees: 2000
No. of variables tried at each split: 1

OOB estimate of  error rate: 32.41%
Confusion matrix:
large medium small class.error
large    238     17   237  0.51626016
medium    80     25   322  0.94145199
small     73     30  1320  0.07238229

Overall Statistics

Accuracy : 0.7297
95% CI : (0.7112, 0.7476)
No Information Rate : 0.8049
P-Value [Acc > NIR] : 1

Kappa : 0.426
Mcnemar's Test P-Value : <2e-16

Statistics by Class:

Class: large Class: medium Class: small
Sensitivity                0.7087       0.84211       0.7294
Specificity                0.8868       0.83981       0.8950
Pos Pred Value             0.5488       0.14988       0.9663
Neg Pred Value             0.9400       0.99373       0.4450
Prevalence                 0.1627       0.03245       0.8049
Detection Rate             0.1153       0.02733       0.5871
Detection Prevalence       0.2101       0.18232       0.6076
Balanced Accuracy          0.7977       0.84096       0.8122


I interpret this output as saying that the model has a 73% accuracy, and that the classifier makes a lot of mistakes for medium and large, but gets small mostly right. Does the P-value indicate that the model is not significant?

Assuming that this precision is OK for my context, how can I validate this model beyond these simple observations?

• Accuracy is a bad metric for classification problems. The p-value isn't calculating the significance of the model. You should make a classification matrix. Dec 11 '19 at 3:18

First of all, if you are trying to impute missing values with a RF model then take a look at the rfImpute() function.
Second, your data in unbalanced, which is why your classification is not good, your model is biased towards the majority class (small) and so it classifies a lot of you cases into the majority class. The issue of imbalance needs to be addressed.