I run XGBoost regression with tree as base learner. I have over 400 variables and more than 30000000 samples. I have generated most important features and was surprised to see that one feature is dominating more than the rest, it was surprising because this variable does not explain variability of the response as they are not connected, I am trying to predict the sold item quantities, I found the most important feature , which is dominating, is the product weight which is not related to the respons at all. What do you think is going on. I run R squared between observed and predicted values and the out out was close to 0.99.

I know R square is not accurate for non linear regression model but what could be the reason of this high value although I see only one variable dominating most important features

  • 2
    $\begingroup$ Isn't this pretty reasonable? Small items tend to be bought in larger quantities than large items and your model is picking up on that. $\endgroup$ – Simon Larsson Jan 21 at 2:22
  • 2
    $\begingroup$ Target leak? if 1 orange weighs 100g, and you're predicting how many oranges were bought given someone bought 1000g, well, you've pretty much encoded the answer '10' in the input. $\endgroup$ – Sean Owen Jan 21 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.