3
$\begingroup$

I have a dataset that has a lot of missing values that it's not viable to drop the rows and it's also not viable to impute with mean/median since it's quite a significant portion of the data and I don't want to add too much bias.

Here's what the matrix of missing values look like. enter image description here

The data has 1,115 rows and the missing data ranges from 150 - 700 missing values. There's a reason for the missing data. The columns with missing data are for measurements of people during a certain event. A lot of people will not have the complete set of measurements because of time and space constraints during the event, so most people will have some of the measurements but not all. Hence, the missing values.

So, I would like to impute the missing values with an "impossible" values, like -1 for the height column.

Would there be a consideration to not do this and instead just leave the NaN values be?

Improve this question
$\endgroup$
2
  • $\begingroup$ Are the missing values in white? $\endgroup$
    – Nicolas Martin
    Sep 19, 2022 at 17:59
  • $\begingroup$ Yes, it is in white. @NicolasMartin $\endgroup$
    – EddieStan
    Sep 20, 2022 at 4:58

1 Answer 1

Reset to default
2
$\begingroup$

Some models, like XGBoost, accept NaN values. So if you use them, you could just keep them like that. I wouldn't use impossible values, like -1, because any type of model WILL interpret it as a valid value (at least it's the case of GB trees, GLMs, Naive Bayes and SVMs)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.