I have a dataset that contains information regarding diabetes patients, like so:
id diabetes diet insulin lifestyle 0 No NaN NaN NaN 1 Yes Yes Yes NaN 2 No NaN NaN NaN 3 Yes NaN NaN NaN 4 Yes Yes NaN Yes 5 Yes Yes Yes Yes
Features diet, insulin and lifestyle have a high percentage of missing data (around 95% each). So initially, I excluded these features from my dataset. However, after taking a closer look at the data, I found the values for diet, insulin and lifestyle to be associated with the value for diabetes feature. This makes sense as diabetes patients would be recommended treatment relating to diet, insulin intake and lifestyle changes.
So in cases where diabetes='No', values for features diet, insulin and lifestyle are missing.
And in cases where diabetes='Yes', I have found that in most cases, at least one feature from diet, insulin and lifestyle to have a value of 'Yes', and the remaining values are missing.
After some reading, I believe features diet, insulin and lifestyle are Missing at Random (MAR), and clearly not missing completely at Random (MCAR) as is explained here.
Anyway, so my question is, should the nature of the missing data here change my decision to remove these features from the dataset due to their high percentage of missing values. Or, should I impute the data for these features, by filling in missing values with "No", like so:
imputer = SimpleImputer(strategy='constant', fill_value='No') x[:, 2:5] = imputer.fit_transform(x[:, 2:5])