How to decide the number of trees parameter for Random Forest algorithm in PySpark MLlib?

Question

I am working on Random Forest algorithm in PySpark MLlib and have a doubt regarding the number of trees parameter that we pass to the model. The standard format of Random Forest modeling in PySpark MLlib is:

model = RandomForest.trainRegressor(trainingData, categoricalFeaturesInfo={},
                                    numTrees=3, featureSubsetStrategy="auto",
                                    impurity='variance', maxDepth=4, maxBins=32)

The doubt that I have is how to decide the optimum value of trees to pass to numTrees parameter? I assume the more the number of trees better should be the performance but would it keep on improving with increase in number of trees? Is there a point after which it will start to have negative impact in performance? If yes then how can I find the optimum number of trees for my dataset?

Answer 1

Maybe you will start to have negative impact of performance by increasing the number of trees. But definitely, at some point, increasing the number of trees will not add much accuracy in your model.

The usual way is to perform a k-fold cross-validation for different number of trees (and any other combination of model's parameters) and choose the one with the best performance. If you have any question about cross-validation, check this page. Instead of using simple accuracy, try to use other metrics that take into account the complexity of the model, like AIC.

Obs.: the final accuracy of the algorithm should be calculated using a separate dataset, not used in the cross-validation (usually called test set).