I am trying to tune a Regression gradient boosting model where my target variable is zero inflated (80% zero) and the rest of the values are distributed as positive and negative values (not necessary symmetrically). What are good practices when training a model like this?

Any specific issues which I should be aware of to generate a good model? Based on my research, Tweedie Gradient Boosting is not fitted for this model because my target variable has a mix of negative and positive values around the zero mode; therefore, it doesn’t follow a tweedie distribution.

  • $\begingroup$ My first thought is that gradient boosted trees are sufficiently flexible that you can just treat this as regression without much customization. My second thought is that you might want a customized loss function, if a node with a few nonzero samples should be predicted as zero rather than very slightly positive. $\endgroup$ – Ben Reiniger Mar 31 '20 at 2:38
  • $\begingroup$ @BenReiniger the problem I am encountering is that the model returns a constant value as a prediction (i.e.,, majority class, converging to the mean) rather than a range of predicted values which are close to zero. I have MSE as the loss function. $\endgroup$ – thereandhere1 Mar 31 '20 at 3:38
  • $\begingroup$ Oh, a constant is surprising. What do you mean by majority class here though? $\endgroup$ – Ben Reiniger Mar 31 '20 at 16:24
  • $\begingroup$ By majority class I am referring to the majority of zero outcome values. $\endgroup$ – thereandhere1 Mar 31 '20 at 18:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.