6
$\begingroup$

Or in other words, data for category A is irrelevant for category B. So it is not present, how can imputing missing data distort/effect learning models broadly. I can't find any logic how to deal with this relative data. So I am sorry that I don't show any effort.

In the following example, geographical zone is only present for Gaz entries.

Data sample:

enter image description here

$\endgroup$
  • $\begingroup$ Depends on what's your task. What do you want to do with the data? Add more info in the question! $\endgroup$ – Francesco Pegoraro Sep 20 '18 at 21:36
  • $\begingroup$ show cross-table produit and zone. because your problem is not clear yet. $\endgroup$ – parvij Oct 21 '18 at 4:56
6
$\begingroup$

There are three types of missing data: Missing Completely At Random (MCAR), Missing At Random (MAR) and Missing Not At Random (MNAR).

Your case is the second, where according to wikipedia it:

occurs when the missingness is not random, but where missingness can be fully accounted for by variables where there is complete information

This means that the presence or not of entries in zone can be derived from the column Produit.

Because the values aren't missing completely at random, normal imputation techniques (e.g. fill with the most common value) shouldn't be applied. Instead I'd recommend treating the missing values as their own category. Just create a category (let's say not available) and fill the missing with this value. From a data science view this makes more sense.

$\endgroup$
  • 1
    $\begingroup$ Missing Not At Random (MNAR) really makes sense, but in this specific case, I'm afraid making "not available" adds information, which distort real life state that learning data is trying to represent, because of the following: Zone is saying something about Produit, when it's gaz, but saying nothing about it when it's elec. Saying not available creates another categorical value that says something, in the same level of importance of other values. So I am not fully agreeing on how to deal with Missing Not At Random (MNAR) in this case $\endgroup$ – bacloud14 Sep 19 '18 at 23:02
  • 1
    $\begingroup$ Actually, I think the variable is MAR (I made a typo in the post). The thing about imputing is that you don't do it to elevate the representation of your data, but to avoid throwing those samples (which would be necessary if you didn't impute them). It's a lesser of two evils type of thing. And I think that since the missing data has a pattern you can use this to your advantage. $\endgroup$ – Mnng Sep 20 '18 at 21:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.