1
$\begingroup$

I have read that LightGBM handles missing values defaultly. And there certain parameters to change the consideration of missing values like zero_as_missing etc.., I have seen some people using negative values (-1, -999) in the place missing values.

So, my question is, which is better 1) leaving it to model to handle or 2) manually replacing the values with negative values?

Thanks in advance

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

The default behavior allows the missing values to be sent down either branch of a split. Replacing with a negative value that is less than all your data forces the (originally) missing values to take the left branch, and so your model has (slightly) less capacity. That may be a good or a bad thing, depending on where you land on the bias-variance curve.

So, I think the best answer here is "it depends on your data." If your missing values actually behave like lesser values, then encoding them as large negative numbers enforces that, reducing capacity in a probably-beneficial way. But without digging into exploratory data analysis, or complex imputation methods (MICE), I'd personally stick to the default behavior.

Improve this answer
$\endgroup$
5
  • $\begingroup$ if the feature has negative values. let say the range would be (-100, 100). What might be the good way to handle missing values replace them or leave them? $\endgroup$
    – Divya reddy
    Commented Sep 18, 2020 at 6:07
  • 1
    $\begingroup$ Everything I said above still applies, just make sure that if you're going to impute with a fixed negative number, it's past -100. Still the distinction between leaving the values as missing or fixed-value imputing lies with your data: do the missing values behave like lesser values? $\endgroup$
    – Ben Reiniger ♦
    Commented Sep 18, 2020 at 14:18
  • 1
    $\begingroup$ how can I know if missing values behave like lesser values? $\endgroup$
    – Divya reddy
    Commented Sep 18, 2020 at 14:38
  • $\begingroup$ "Behave like" is a very rich phrase, but for a good start, get a histogram of the target value by this feature, with a separate category for "missing". $\endgroup$
    – Ben Reiniger ♦
    Commented Sep 18, 2020 at 14:48
  • $\begingroup$ sorry for the late reply! (value - target - percent) nan - 0 = 7%; nan - 1 = 6.35%. can you explain anything with this? $\endgroup$
    – Divya reddy
    Commented Sep 19, 2020 at 8:38
0
$\begingroup$

You have two simple choices:

  • remove the examples with missing values (I know not always possible but just for the sake of being exhaustive)

  • use an imputer i.e. replace it with a value that would minimise the noise.You have several choices:

  1. mean/median imputing (you can replace the missing value by the mean or median over this particular feature
  2. cluster-based imputing: this would mean training another (unsupervised) model on features that are not missing and assign missing values based on the closest cluster learned
Improve this answer
$\endgroup$
1
  • $\begingroup$ there is problem with both ways in my case. 1)if i remove the samples with missing values I'll lose lot of data and 2) if i replace them with mean or mode or something, samples meaning will be lost $\endgroup$
    – Divya reddy
    Commented Sep 18, 2020 at 6:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.