So I understand the main difference between Random Forests and GB Methods. Random Forests grow parallel trees and GB Methods grow one tree for each iteration. However, I am confused on the vocab used with scikit's RF regressor and xgboost's Regressor. Specifically the part about tuning for the number of trees/iterations/boosting rounds. From my understanding each of those terms are the same. They determine how many times a decision tree should be calculated based on the algorithm. However, should I be referring to them as ntrees or n_estimators? Or should I simply use early stopping rounds for my xgboost and tune the number of trees only for my rf?
My Random Forest:
rf = RandomForestRegressor(random_state = 13) param_grid = dict(model__n_estimators = [250,500,750,1000,1500,2000], model__max_depth = [5,7,10,12,15,20,25], model__min_samples_split= [2,5,10], model__min_samples_leaf= [1,3,5] ) gs = GridSearchCV(rf ,param_grid = param_grid ,scoring = 'neg_mean_squared_error' ,n_jobs = -1 ,cv = 5 ,refit = 'neg_mean_squared_error' )
model = XGBRegressor(random_state = 13) param_grid = dict(model__ntrees = [500,750,1000,1500,2000], model__max_depth = [1,3,5,7,10], model__learning_rate= [0.01,0.025,0.05,0.1,0.15,0.2], model__min_child_weight= [1,3,5,7,10], model__colsample_bytree=[0.80,1] ) gs = GridSearchCV(model ,param_grid = param_grid ,scoring = 'neg_mean_squared_error' ,n_jobs = -1 ,cv = 5 ,refit = 'neg_mean_squared_error' )