0
$\begingroup$

Let's suppose that I have a dataset with 5 numerical features of which each of them has some missing values and all of them have only non negative values.

Some suggested ways to deal with missing data are:

  1. Remove the rows which have even one missing value
  2. Impute the missing values

I do not prefer (1) because then you miss some valuable information from the rest of the features for these rows.

I do not prefer (2) because in general (it depends on the application) it introduces quite a lot of noise in the data.

What I am thinking is to: 3. Replace the missing values with a unique value (eg -1 or -999)

As I said, in my example the features have only non-negative numbers so values such as -1 or -999 will be only encountered by the algorithm for missing data.

What are your thoughts on (3)?

What are the advantages and disadvantages of this approach?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

This depends on the exact learning algorithm which is used, but most of the time numerical values are treated... numerically.

It's easy to see with the case of linear regression: let's say a feature has values between 0 and 100, but with a few missing values. Now replace missing values with -999: the coefficient learned for this feature will be completely different and based on meaningless indications, so the performance will decrease.

This idea might work in the case where the numerical values are 'binned' into several classes, i.e. transformed into ordinal/categorical variables.

But the best way to use such values is to use an algorithm and framework which can deal with missing values in the first place.

Improve this answer
$\endgroup$
10
  • $\begingroup$ Thank you for your reply(upvote). Yes, I agree that it will be probably a quite problematic for linear models but what about non-linear models (eg random forest)? Will it be relatively ok? $\endgroup$
    – Outcast
    Oct 18, 2019 at 14:45
  • $\begingroup$ In any case, how to handle missing values is a quite big topic and I have not found any better way to do this (I am not in favor of imputations). $\endgroup$
    – Outcast
    Oct 18, 2019 at 14:46
  • $\begingroup$ @PoeteMaudit technically your approach is a kind of imputation. Unfortunately it's annoyingly difficult to find information about how a particular implementation of an algorithm treats missing values, but many of them are able to handle missing values in a smart way. That's the case for (some?) decision trees implementations. See for example how different algorithms deal with missing values in Weka here: weka.8497.n7.nabble.com/… $\endgroup$
    – Erwan
    Oct 18, 2019 at 15:00
  • $\begingroup$ imputation->"the assignment of a value to something by inference from the value of the products or processes to which it contributes" . It is no inference (at least based on the value of other features) what I do by assigning a constant but based on the (missing) value of the feature itself but ok I see your point. $\endgroup$
    – Outcast
    Oct 18, 2019 at 15:34
  • $\begingroup$ At the main question now, my question was about non-linear models (eg random forests) in general. So I would like to know your opinion on them too (and not only about specific implementations of them) :) . Personally, I use the standard implementation of random forest of SkLearn in any case. $\endgroup$
    – Outcast
    Oct 18, 2019 at 15:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.