I am training an XGboost model for binary classification on around 60 sparse numeric features. After training, the feature importance distribution has one feature with importance > 0.6, and all the rest with importance <0.05.
I remove the most important feature, and retrain. The same distribution forms; the most important feature has importance > 0.6, and the rest have < 0.05. I continued to remove the most important feature and retrain, remove and retrain, remove and retrain, etc. My f1-score started to drop, but every time there was one feature more important than the rest.
Also worth noting, when I removed the most important feature and retrained, the new most important feature was not the second most important feature from the previous training.
I cannot explain this behaviour intuitively. Does anyone know why this pattern arises?