What is the difference between Missing at Random and Missing not at Random data?

Question

I have been working with a dataset where the missing data seem to following a few particular patterns. I have gone through a lot websites and articles related to missing data but I haven't been able to understand the difference between MAR and MNAR.

First I would like to give the description of the dataset and the patterns of the missing data. So it would be easier for you to explain the difference between the two and also it would enable to identify the what are those patterns in my dataset.

Description of the dataset: It is a transaction history dataset of a cycle company(Imaginary company) which contains descriptions about customers like their name, DOB, gender, geographic location, Income, social status and the transaction details like the cycle brand name bought, size of the cycle(small,M,L), type of cycle like (Mountain, Road and Standard), Product manufacturing cost, List Price of the product and finally the transaction date.

Pattern in the missing data: There are basically four different patterns of missing data that I have identified in the dataset:

Pattern 1: This is related to product details in the transaction history: Whenever there is some missing data about say the brand name then the details regarding Product line, Product class, Product size, standard cost and Product_first_sold_date are also missing but the list price alone is available but then these list prices seem unique as they cannot be found repeating in the same column. This could blamed on the company for not recording the details properly but there is actually no such company that exists and it is upto me to deal with.

Pattern 2: Whenever there is missing data about a customer say first name then all the other columns about the customer are also missing like Gender, Past_3_years_bike_related_purchases,Job_Industry_Category, Wealth_Segment,Deceased_Indicator and Owns_Car. Only the customer ID in this case is available. Probably this could be taken as the customer didn't want to disclose the details but then again it is upto me to deal with.

Pattern 3: Whenever there is some missing data about the geographic location of the customer like say the Address then the Post code, state,country and property evaluation column data are also missing but the general customer details like name, gender, DOB are all available. Probably in this case the customer didn't want to reveal their geographic details and I have got to deal with it.

Pattern 4: Most interesting one: The gender column has three categories M, F and U. U can be taken as not disclosed. Whenever the gender is U then their age and tenure are also missing. This could be taken as those not ready to disclose their Gender are not ready to disclose their age and tenure.

Sometimes some patterns occur together.

What category of missing date like(MAR, MCAR and MNAR) do these fall into? And how do I deal with it. Any suggestion would be extremely helpful. Thanks.

Answer 1

Definitions of missingness process are tricky. Missing completely at random occurs when the missingness is really at random (MCAR; e.g. when conducting a survey there are error in the data entry process).

• Missing at random (MAR) occurs when the missingness is not really at random, but when it could be considered at random conditioning on what is observed in the rest of the data (e.g. males are less likely to express their opinion in a survey but this is completely not related to their attitude as customers).
• Missing completely at random occurs when the missingness is really at random (MCAR; e.g. when conducting a survey there are error in the data entry process).
• Missing not at random (MNAR) occurs when the missingness depends on the value of the variable (those who buy more tend to not answer survey questions).

I think that it is difficult to figure out in what category are your missing data patterns. In your case seems to me that MAR is a reasonable assumption (which is impossible to test statistically) and that multiple imputation with chained equations could be an option (mice or miceadds in R).

If you adopt a mice approach, a crucial part in the application is to carry out sensitivity analyses. what happens if you: - impute only some of your missing data pattern? - vary the number of imputed sets - perform a complete case analysis.

Answer 2

I'm not sure about classifying each pattern as MAR / MCAR / MNAR.

Once you've assessed your data and you would like to impute the missing values using functions and models, then there is a useful R package called mice ("Multivariate Imputation by Chained Equations").