In the case of having a combination of categorical and numerical Attributes, I usually convert the categorical attributes to one hot vectors. My question is do I leave those vectors as is and scale the numerical attributes through standardization/normalization, or should I scale the one hot vectors along with the numerical attributes?
$\begingroup$
$\endgroup$
asked May 14, 2018 at 17:54
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
2
Once converted to numerical form, models don't respond differently to columns of one-hot-encoded than they do to any other numerical data. So there is a clear precedent to normalise the {0,1} values if you are doing it for any reason to prepare other columns.
The effect of doing so will depend on the model class, and type of normalisation you apply, but I have noticed some (small) improvements when scaling to mean 0, std 1 for one-hot-encoded categorical data, when training neural networks.
It may make a difference too for model classes based on distance metrics.
Unfortunately, like most of these kind of choices, often you have to try both approaches and take the one with the best metric.
answered May 14, 2018 at 18:11
-
1$\begingroup$ The wording was a bit unclear. Are you saying you only normalize one-hot-encoded columns if you've normalized any non-o.h.e. columns? $\endgroup$– Info5ekCommented Feb 14, 2019 at 2:21
-
2$\begingroup$ @Info5ek: I am saying that it might be better to normalise one-hot-encoded columns, and if you are already doing it for other columns then you may as well give it a try. There are no fixed rules to this, too much depends on the problem at hand. $\endgroup$ Commented Feb 14, 2019 at 7:50