7
$\begingroup$

Or in other words, data for category A is irrelevant for category B. So it is not present, how can imputing missing data distort/effect learning models broadly. I can't find any logic how to deal with this relative data. So I am sorry that I don't show any effort.

In the following example, geographical zone is only present for Gaz entries.

Data sample:

enter image description here

Improve this question
$\endgroup$
2
  • $\begingroup$ Depends on what's your task. What do you want to do with the data? Add more info in the question! $\endgroup$
    – Francesco Pegoraro
    Commented Sep 20, 2018 at 21:36
  • $\begingroup$ show cross-table produit and zone. because your problem is not clear yet. $\endgroup$
    – parvij
    Commented Oct 21, 2018 at 4:56

1 Answer 1

Reset to default
7
$\begingroup$

There are three types of missing data: Missing Completely At Random (MCAR), Missing At Random (MAR) and Missing Not At Random (MNAR).

Your case is the second, where according to wikipedia it:

occurs when the missingness is not random, but where missingness can be fully accounted for by variables where there is complete information

This means that the presence or not of entries in zone can be derived from the column Produit.

Because the values aren't missing completely at random, normal imputation techniques (e.g. fill with the most common value) shouldn't be applied. Instead I'd recommend treating the missing values as their own category. Just create a category (let's say not available) and fill the missing with this value. From a data science view this makes more sense.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Missing Not At Random (MNAR) really makes sense, but in this specific case, I'm afraid making "not available" adds information, which distort real life state that learning data is trying to represent, because of the following: Zone is saying something about Produit, when it's gaz, but saying nothing about it when it's elec. Saying not available creates another categorical value that says something, in the same level of importance of other values. So I am not fully agreeing on how to deal with Missing Not At Random (MNAR) in this case $\endgroup$
    – bacloud14
    Commented Sep 19, 2018 at 23:02
  • 1
    $\begingroup$ Actually, I think the variable is MAR (I made a typo in the post). The thing about imputing is that you don't do it to elevate the representation of your data, but to avoid throwing those samples (which would be necessary if you didn't impute them). It's a lesser of two evils type of thing. And I think that since the missing data has a pattern you can use this to your advantage. $\endgroup$
    – Mnng
    Commented Sep 20, 2018 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.