XGBoost most important features appear in multiple trees multiple times

Question

I am fitting xgboost model (scala-spark) to my dataset of transactions. I have about 2 millions of transactions in my training set which is highly unbalanced with a ratio of positive/negative<0.001 classes. I have now about 300 features in the model.

Then I take an output model and count for each feature

• In how many trees a feature was present
• In how many splits a feature was present

Then I sort the features from the ones with most trees to least trees. I get some results that I am not sure of. Features at the top of the list with most trees and splits basically appear in each single xgboost tree multiple times. For example in xgboost with 100 rounds and colsample_bytree=1.0 and max_depth=6 I would see a feature A appear in 100 trees and in ~400 splits. Feature B appears in 98 trees and 350 splits etc... Basically it seems that all my trees are based on all the same same top features in different configurations.

Question:

Does it mean:

• It normal
• Do those features overfit my model
• I tried to force model to take other features by decreasing colsample_bytree or colsample_bylevel and it helps but somewhat, but model performance does not dramattically improve.

Any other suggestions?

Update Observations (Feb 2019)

• The features are continuous not categorical
• When I sort all features based on their total gain (sum of gain in all nodes that split on the feature), the features that appear in all trees multiple times can have highest total gain or quite low total gain. There is not rule here.

Answer 1

To your questions:

1. It can be normal.
2. Features itself are not responsible for overfitting. It can be noise from them or just correlation without causation. I.e. Feature correlates in the training set, but not in the test set. Overfitting is related to models.
3. Maybe you still don't have a feature that would improve your model significantly. It's hard to say with such limited information. You need to try some experiments with feature selection.