Feature engineering for categorical variables

I have some categorical variables in my dataset for a regression problem.

1) One of the variable can take 3 values (Girls, Boys, Girls&Boys). Converting it into one-hot encoding or binary encoding will treat all three values as a different class. How can I use it efficiently retaining the information that 'Girls&Boys' include both? Is breaking into two separate columns for girls and boys is the only approach in such cases?

2) Age range: (18-35, 35-50 etc.) I have broken it down to 2 columns of age_min and age_max. Is there any better way to use such kind of features with values in range?

3) Range of percentage: It can take only 5 values (0, 1, 1-5, 5-10, 10). How should I use this variable for training my model? Here I cannot break it down into 2 columns (like with age) because of "1" and "10" fixed values. How to treat variables with fixed as well as range values?

4) Similarity: It can also take 5 values (0, 1, 1-5, 5-10, 10) but it also has an "auto" option. That means if it is "auto" it can take any random value which we do not know about. How should I incorporate that? Should I create a separate column for "whether it is auto or not"? Then, in the original similarity column, what value should I put for observations with "auto". I cannot put "0" as it is already a value and I am sure that "auto" won't be 0. Does putting it "None" will create a difference? How to treat unknown values?

• Are you familiar with Entity Embedding for Categorical Encoding in Deep Learning? Simply you can include your categorical features implicitly during training in a deep learning model for regression or classification. I have prepare a tutorial on this for regression actually, see github.com/mmortazavi/EntityEmbedding-Working_Example. Jun 20 '19 at 5:33

Before I begin let me note that there isn't a right and a wrong way to do feature engineering. If you want to be thorough, you need to try things and see if they work or not.

Now let me give you my opinions (i.e. how I might approach things), but they aren't guaranteed to work:

1) One of the variable can take 3 values (Girls, Boys, Girls&Boys). Converting it into one-hot encoding or binary encoding will treat all three values as a different class. How can I use it efficiently retaining the information that 'Girls&Boys' include both? Is breaking into two separate columns for girls and boys is the only approach in such cases?

I think one-hot encoding works best here. Depending on what information does Girls&Boys store you can select how to encode it. If Girls&Boys == 1 if both Girls and Boys are 1, then I think you can safely omit it.

2) Age range: (18-35, 35-50 etc.) I have broken it down to 2 columns of age_min and age_max. Is there any better way to use such kind of features with values in range?

What you've done isn't bad. Another option would be to treat this like an ordered variable and simply map each range to a number (while preserving the order). E.g. 18-35 --> 1, 35-50 --> 2, etc.

3) Range of percentage: It can take only 5 values (0, 1, 1-5, 5-10, 10). How should I use this variable for training my model? Here I cannot break it down into 2 columns (like with age) because of "1" and "10" fixed values. How to treat variables with fixed as well as range values?

Same thing as before. I'd treat this like an ordered variable. You could even leave 0 and 1 as they are. The rest would go like this: 1-5 --> 2, 5-10 --> 3, 10 --> 4. If some of these 5 categories has very few values you could merge it with a neighboring one!

4) Similarity: It can also take 5 values (0, 1, 1-5, 5-10, 10) but it also has an "auto" option. That means if it is "auto" it can take any random value which we do not know about. How should I incorporate that? Should I create a separate column for "whether it is auto or not"? Then, in the original similarity column, what value should I put for observations with "auto". I cannot put "0" as it is already a value and I am sure that "auto" won't be 0. Does putting it "None" will create a difference? How to treat unknown values?

This is trickier here. One thing I'd do is what you mention: to have an auto column which can be either 1 or 0 if the user chose "auto" or not. However, it's tricky to encode the range now, because how would you encode those that selected "auto"? One way is to make a separate category (e.g. auto --> 5), but this wouldn't correct for an ordinal variable. Another option would be to one-hot encode this. Your choice I think depends on two reasons: a) Given the "auto" option how important is it preserving the range information as is? Could you group categories together? could you disregard their order? b) How many of the samples are "auto"? If they are very few it might not even make a difference. If they are too many then encoding the range might have no meaning.