Continuous variables with missing values of type MNAR

Question

I have continuous variables (weather features) with missing values of type MNAR (a different distribution with and without missing values). I learned that these variables should be transformed into categorical by adding a 'missing' category. Filling with mean/median/mode is not an option in this case (although I saw in different projects online that even though there are MNAR values, some still impute them with mean/median, contrary to what I have learned).

For example: Cloud coverage feature with MNAR missing values converted to categorical feature with subcategories of: ['very high', 'high', 'medium', 'low', 'very low', 'missing'] (see image):

I want to check seasonality and trend (time series data) and add those features to the dataset, but when I check it before converting the variables it simply gives me more variables with missing values that lead to the same problem of MNAR.

import statsmodels.api as sm

df.set_index('timestamp', inplace=True)
analysis = df[['cloud_coverage']].dropna().copy()
s = sm.tsa.seasonal_decompose(analysis, period=30)

df['trend_cloud_coverage'] = s.trend
df['seasonal_cloud_coverage'] = s.seasonal
df['residual_cloud_coverage'] = s.resid

All these 3 new features contain missing values corresponding to the missing values in cloud_coverage feature.

How do I deal with missing values in this case where time series is of importance?

Thank you all for any help.

Answer 1

I'm not sure which is your concrete question.

However I would recommend you use KNNImputer. This function will change the "missing values" using the knn algorithm. Which means that it will input values focusing on other similar samples.

To do that, you will probably first to LabelEncoder your column keeping NaN (since your values is ordinal, you won't have problem). And the apply then KNNImputer.

If you want to just obtain values input without decimal, set the number of neighbors to just 1.