Missing data imputation with KNN

Question

-1 down vote favorite I have a dataset including missing data for most of the variables. Assume the dataset is as follows:

Obs.  var1  var2  var3  var4  var5  var6
1     x11   x12   x13   x14    Nan  Nan
2     x21   x22   x23   Nan    x25  x26
3     x31   x32   x33   x34    x35  x36
...
n     xn1   xn2   xn3   xn4    Nan  xn6

I have split the dataset to d1 where we have complete data for all variables and d2 where all records have at least one missing variable.

I made different models using KNN: To predict the values of var5 and var6 for the first observation, I used d1 (dataset without missing value) and modeled on var1, var2, var3 and var4.

To predict the value of var5 for the last observation, I used d1 and modeled on var1, var2, var3, var4, and var5.

Does my approach make sense?! Any suggestion are welcome. Thank you.

Answer 1

There are various approaches for dealing with missing values. Suppose we've got 4 instances in a dataset:

x1 = [1 2 3]
x2 = [1 ? 3]
x3 = [2 4 2]
x4 = [1 3 3.5]

1. one simple approach (specially popular in medical datasets) is finding values with regard to most similar instance; in the above case, missing value of x2 would be then: 2 (since x1 is the most similar)

2. a more sophisticated approach is weighted averaging through k most similar instances (of course only applicable if the missing value is numeric or at least ordinal) in this case, you should calculate: x2(2) = (2*(1-(0/3)) + 3*(1-(0.5/3)) / (2-(0/3)-(0.5/3)) (x4 has also been counted as a similar case

3. another approach, is voting among k most similar instances (applicable both for categorical and ordinal values).

cases 1 and 3 are what you have implemented (case1 = 1 Nearest Neighbor / case2 = k-nearest neighbors)

there are other approaches for handling a missing value, but it depends on what you're going to do with your dataset. For example in very large datasets, sometimes it is efficient to simply ignore every instance that contains at least one missing value, or ignore only the missing value (not the entire instance) in further processes (e.g. in VFI algorithm)