3
$\begingroup$

I try to evaluate several NA imputation methods with supervised approach: I clone my original data frame with no NAs, artifically insert NAs into the resulting Data Frame and apply imputations to the latter.

Now, I'd like to evaluate the imputations by comparing the imputed new DFs with the original one. I wonder what would be the best metod; is there any distance method for instance that I could apply to the original/imputed DF pairs?

(My DF contains only numeric data, but a solution that handles factor variables as well would be especially handy).

$\endgroup$

2 Answers 2

1
$\begingroup$

If we want to check the imputation efficiency, we should probably compute the performance for the imputed values only. Indeed, the performance depends on the percentage of missing values.

$\endgroup$
0
$\begingroup$

It's not necessarily a distance but what you could do is the following schema, for one column of potential NAs. Let's call that one $x_n$, the rest are knowns in any case. We will have three options, keep the full original (best case scenario), impute the artificially added NAs in multiple ways (realistic scenario) and drop the whole column (pessimistic scenario). Now we know how well the best case scenario works and what happens if we just drop the whole column. We just measure what we want to optimize, MSE for example. Our imputation methods will normally have a performance between best case and worst case. Then you could quantify your imputation method by putting it on a line between worst and best case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.