I was recently playing around with the FICO explainable machine learning challenge dataset.
In the dataset, there are a bunch of numerical features which have values values typically in the 0-100 range. However, some features can have special values of -9, -8 or -7 for some rows. I was wondering what the best way to encode such a dataset would be?
What I ended up doing was to create F_cat_-9, F_cat_-8, F_cat_-7, F_cat_0 for all such features, and pass the new features to the model along with the original feature F. This improved performance for logistic regression, but surprisingly not for xgboost models. I am still a little concerned that the xgboost model might be using the original feature with special values in an unwanted way.
Row 1: F1: -8, F2: 22
Row 2: F1: 2, F2: 15
New Row1: F1_cat_-8: 1, F1_cat_0: 0, F1_cat_-9: 0, F1: -8, F2: 22
New Row2: F1_cat_-8: 0, F1_cat_0: 1, F1_cat_-9: 0, F1: 2, F2: 15