# Does increasing the n_estimators parameter in decision trees always increase accuracy

I'm using some ML algorithms (from sklearn lib) and on most of them there is a parameter n_estimators which is (if I understood well) the number of used trees. Intuitevely I would say that the more trees you use, the more accurate results you get. It turned out to be not exactly true, sometimes, a very few number of trees give much better results, but I can't figure out why ?

Edit

Some precisions: this is a regression problem, with a dataset containing about 4000 samples and 500 features. I'm using GradientBoostingRegressor, ExtraTreeRegressor, AdaBoost, RandomForest.

Edit 2

Additional info: I use a cross-validation with KFold=10 to evaluate the accuracy of the algorithms. The best n_estimators value seems to be 50, which give a R2 score of ~56/57% +- 8% for all above cited algo. When I try to increase it, the score quickly decreases. I tried several values, from 100 to 500, it keeps decreasing even reaching 52%.

• What kind of dataset are you working on? Are you using RandomForestClassifier, ExtraTreesClassifier, or something else? What kind of values are you using for n_estimators? Jun 22, 2017 at 3:41
• Thanks for teh edit. I think we'll need some additional information. How do you know that you get better results with a small number of trees? What experiments have you done, exactly (what was the procedure), and what exactly are the results (the numbers)? Are you using a held-out validation set and test set? How much better? Could it be statistical noise? What values of n_estimators are you using?
– D.W.
Jun 23, 2017 at 0:40