I am running the Kaggle Video Games sales dataset through an XGboost algo.
I want to encode the categorical column of "Game Rating" from E, M, etc. to 0-5
when I use: data = pd.get_dummies(data=data, columns=['Game_Rating])
pandas adds a column to my dataframe for each unique observation in Game_Rating
Rating_E Rating_E10+ Rating_EC Rating_K-A Rating_M Rating_RP Rating_T
1 0 0 0 0 0 0
0 0 0 0 0 0 0
1 0 0 0 0 0 0
1 0 0 0 0 0 0
0 0 0 0 0 0 0
My question is two-fold: Does Pandas have the ability to replace categorical values of a dataframe inline, preserving the original column structure
if not:
I have a (slower) method that does the encoding inline and preserves the column structure of the dataframe. I am trying to keep the dataframe as similiar to the original dataset as possible for future processing. What are the risks I may not be seeing of using my own (inline) encoding method?
df['your column name'].astype('category').cat.codes
. Be careful when doing this, because it implies an ordering on the columns. Your learning algorithm might overfit to spurious relationships that appear in the training data $\endgroup$ – timleathart Oct 6 '18 at 7:58