# Q: xgboost regressor training on a large number of indicator variables results in same prediction for all rows in test

I'm training a XGBoost regressor in Python on a data set with a large number of indicator variables (one-hot-encoded from categorical variables) and a few numerical variables.The dataset size is over a million rows with a total column number of ~1000. The parameter I used was:

param = {
'objective':            'reg:linear',
'bst:max_depth':        6,
"min_child_weight":     1,
'gamma':                5,
'max_delta_step':       1,
'bst:eta':              0.01,
'verbose':              1
}
num_rounds = 1000


I checked the param of fitted trees and seems like most of them have a depth of 1 or 2. This happened for this feature size of ~1000; for a smaller feature size, the prediction result seems OK (but theoretically ~1000 would provide a better prediction result, so that's why I want it to work).

Does this mean that there's only very few variables are used in the tree construction and results in most of the trees being identical? Or am I doing anything very stupid?

The dataset is quite large and I'm not sure how to provide a runnable sample. Any possible suggestions or general discussions are welcomed!