# Should I use pandas get_dummies and create additional columns or use my own encoding code that keeps 1 column?

I am running the Kaggle Video Games sales dataset through an XGboost algo.

I want to encode the categorical column of "Game Rating" from E, M, etc. to 0-5

when I use: data = pd.get_dummies(data=data, columns=['Game_Rating])

pandas adds a column to my dataframe for each unique observation in Game_Rating

Rating_E  Rating_E10+  Rating_EC  Rating_K-A  Rating_M  Rating_RP  Rating_T
1            0          0           0         0          0         0
0            0          0           0         0          0         0
1            0          0           0         0          0         0
1            0          0           0         0          0         0
0            0          0           0         0          0         0


My question is two-fold: Does Pandas have the ability to replace categorical values of a dataframe inline, preserving the original column structure

if not:

I have a (slower) method that does the encoding inline and preserves the column structure of the dataframe. I am trying to keep the dataframe as similiar to the original dataset as possible for future processing. What are the risks I may not be seeing of using my own (inline) encoding method?

• What does your own encoding do? It's hard to give recommendations without knowing anything about it Oct 5 '18 at 15:45
• my encoding method does the following: 1.) creates a list of unique (categorical) values in a column. 2.) creates an index of 0 to 'x' to represent each unqiue (categorical) value 3.) uses pandas.iloc to replace the unique categorical value with the numerical value. I could summarize by saying it does the same thing as pd.get_dummies - but replaces values inline, leaving column structures in their original state Oct 5 '18 at 15:49
• If I've understood correctly, that's the same as numeric encoding (see scikit-learn.org/stable/modules/generated/…) Oct 5 '18 at 15:56
• You can get the same effect very efficiently in pandas by using df['your column name'].astype('category').cat.codes. Be careful when doing this, because it implies an ordering on the columns. Your learning algorithm might overfit to spurious relationships that appear in the training data Oct 6 '18 at 7:58

## 1 Answer

credit for this answer goes to @timleathart in the comments above:

You can get the same effect very efficiently in pandas by using df['your column name'].astype('category').cat.codes. Be careful when doing this, because it implies an ordering on the columns. Your learning algorithm might overfit to spurious relationships that appear in the training data