First, let's create a mcve to play with:
import pandas as pd
import numpy as np
In : categorical_array = np.random.choice(['Var1','Var2','Var3'],
df = pd.DataFrame(categorical_array,
columns=map(lambda x:chr(97+x), range(categorical_array.shape)))
# Add another column that isn't categorical but float
df['d'] = np.random.rand(len(df))
a b c d
0 Var3 Var3 Var3 0.953153
1 Var1 Var2 Var1 0.924896
2 Var2 Var2 Var2 0.273205
3 Var2 Var1 Var3 0.459676
4 Var2 Var1 Var1 0.114358
Now we can use pd.get_dummies to encode the first three columns.
Note that I'm using the
N-1 dummies are sufficient to fully describe
N possibilities (eg: if
a_Var3 are 0, then it's
Also, I'm specifically specifying the columns but I don't have to as it will be columns with dtype either
categorical (more below).
In : df_encoded = pd.get_dummies(df, columns=['a','b', 'c'], drop_first=True)
d a_Var2 a_Var3 b_Var2 b_Var3 c_Var2 c_Var3
0 0.953153 0 1 0 1 0 1
1 0.924896 0 0 1 0 0 0
2 0.273205 1 0 1 0 1 0
3 0.459676 1 0 0 0 0 1
4 0.114358 1 0 0 0 0 0
In your specific application, you'll have to provide a list of column that are Categorical, or you'll have to infer which columns are Categorical.
Best case scenario your dataframe already has these columns with a
dtype=category and you can pass
columns=df.columns[df.dtypes == 'category'] to
Otherwise I suggest setting the
dtype of all other columns as appropriate (hint: pd.to_numeric, pd.to_datetime, etc) and you'll be left with columns that have an
object dtype and these should be your categorical columns.
The pd.get_dummies parameter columns defaults as follows:
columns : list-like, default None
Column names in the DataFrame to be encoded.
If `columns` is None then all the columns with
`object` or `category` dtype will be converted.