# Treating missing data in categorical features

I have a dataset with one of the categorical columns having a considerable number of missing values. The interesting thing about this column is that it has values only for a particular category in "another" column .

For eg :

column 1                        column2
========================================


My column of interest has values only for one category (Facebook) that is present in another column. Therefore, the missing values for google cannot be imputed with average, cannot be predicted and those rows cannot be ignored either.

In such a situation, is it wise to consider the missing values '-' as a separate category in one-hot encoding? Or will this affect my machine learning model badly?

• To me it depends and you have to make some test both with and without variable. Did you also try to merge column 1 and column 2 variables ? (In your example, you could make 3 variables Google, FacebookImage and FacebookVideo). That's another thing you can try to avoid having 2 highly correlated columns. – BeamsAdept Aug 21 at 14:19

You could break the column 2 from your example into number of columns : Image,Video....

So the new features will be like:

Column1  Image  Video

• We can follow this method for all kinds of categorical columns? – Vikas Ukani Sep 19 at 5:23
• Suppose, There is an categorical feature in which there are too many unique values, For that, This method goes wrong, Right? – Vikas Ukani Sep 19 at 5:24

You can try this:

import pandas as pd

df_new = pd.get_dummies(df, columns=['column2'])
print(df_new)


Output:

    column1  column2_Image  column2_Video

• What if there are many unique values in column_2, For Instance, Image, Video, PDF, DOC, Excel, Audio, etc. – Vikas Ukani Sep 19 at 5:26
• It will work. For example, if you add a new value (email), a new column will be added: column2_Email column2_Image column2_Video – SKM Sep 19 at 6:37