I'm using XGBoost and all its doing is using the feature in the first column of my data. My feature importance chart correlates perfectly to the position of the feature in my xtrain. If I shuffle the columns in xtrain, the feature chart changes along witht he shuffle after re-running the model.
This suggests that XGBoost is "getting stuck" on the first feature. Is there a way to prevent this? I was thinking by limiting how many times the same feature can appear in a tree?