Categorical vs continuous feature selection/engineering

Question

I'm working with a dataset with a number of potential predictors like :

Age : continuous

Number of children : discrete and numerical

Marital Situation : Categorical ( Married/Single/Divorced.. )

Id_User : Categorical ( an id of the user who conducted the first interview with this person )

I'm stopping at four potential predictors, there are more, but for the sake of shortness, these would be enough to ask my question.

Question : Continuous features are easy to deal with, normalize, and feed it to the model, what about categorical and independant ?

Note : I get that categorical features that follow a certain pattern can be encoded as integers and fed to the model, but what if those categorical features have no meaning as integers ( 1 for single, 2 for married , 3 for divorced ; for the model that treats it as a quantitative predictor it doesn't make sense to feed it to it like that)

Any ways to deal with these different types of features?

Answer 1

What you are looking for are called dummy variables, they convert your categorical data into a matrix where the column is 1 if the person belongs to a category or 0 otherwise.

The variable ID is not convertible because you don't want your model to overfit over your ID data (meaning: You don't want your model to remember the result for every ID, you want your model to be general).

import pandas as pd
dataset2 = pd.get_dummies(dataset)

Answer 2

For encoding categorical features, there is two common ways:

Ordinal encoder

This is the way you mentioned as 'encoded as integers'. In this method, an integer starting from 0 is assigned to each category. The problem of this method is that it randomly prioritize categories. So in cases when there is no priority among categories, this encoding is meaningless as you mentioned. The only case it work is when assigning larger integer to some categories is meaningful.

One-hot encoder

This method makes a feature vector (one-hot vector) for each categorical feature which is the same size as the number of categories. The method assigns each component of the vector to one of the categories. For each data sample, it assigns 1 to component which its corresponding category is present at the sample and assigns 0 to other components. The benefit of this method is that unlike ordinal encoder it does not prioritize any category.

So in your case, I highly recommend that you use one-hot encoder.

Answer 3

As others have said, dummy variables is one method. Another method is to take quantitative statistics from the populations having that property. For instance, you can create a "marital situation average" column, and populate it with the average value of the target variable among people with the same marital situation as that subject.

If you are using a tree method, simply assigning integers to each category will approximate dummy variables, especially if there are only a few categories. For instance, if the only categories for marital situation are Married, Single, Divorced, and Widowed, and you assign them 0, 1, 2, 3 respectively, then the only possible splits are Married vs. Everything else, Widowed vs. Everything else, or Married/Single vs Divorced/Widowed. So two thirds of the splits are effectively dummy variables, and the last one will turn into a dummy variable as soon as you split on that variable again.

Answer 4

One possibility to deal with categorical inputs is to introduce the category input vector $\boldsymbol{t}$. The category input vector of the $n^{\text{th}}$ observation is given by

$\boldsymbol{t}_n=[t_{1n}, t_{2n},...,t_{Kn}],$ in which $K$ is the number of categories. If the continuous input vector $\boldsymbol{x}_n$ is belonging to category $k$, then $t_{1i}=1$ for $i=k$ and $t_{1i}=0$ for $i\neq k$.

This type of encoding is called one hot encoding for classification.

Answer 5

There could be a number of ways of handling categorical data but what I have seen so far is to create a numeral mapping of the categorical data and then one-hot-encode the mappings to feed into the neural network.

If you are working with Keras, you can use the to_categorical function to transform your mappings accordingly.

>>> from keras.utils import to_categorical

>>> y = [0,1,0,1,1]
>>> oh_y = to_categorial(y, num_classes=2)
>>> print(oh_y)

[[1,0],[0,1],[1,0],[0,1],[0,1]]
```