1
$\begingroup$

Should we normalize the categorical columns in our dataset? Or just the numerical columns?

$\endgroup$
  • 1
    $\begingroup$ What exactly do you mean by 'normalize categorical columns'? $\endgroup$ – 89f3a1c Jul 22 at 13:37
  • $\begingroup$ I mean after encoding the categorical variables using Target encoding, can we use mean normalization? And would it be useful? $\endgroup$ – Matt Jul 23 at 5:46
1
$\begingroup$

Normalization and standardization are transformations that can only be applied on metric variables. You cannot normalize categorical variables.

Either you label-encode your data or you one-hot-encode it, depending on the algorithm you use. That's all you do with it.

$\endgroup$
  • $\begingroup$ If we one-hot or target encode the categorical variables, will standardization or normalization help the model? Or I can simply not apply any sort of scaling and algorithm works fine anyway? $\endgroup$ – Matt Jul 23 at 5:48
  • $\begingroup$ Normalization projects your values into the range between 0 and 1. If you one-hot-encode them, they are already either 0 or 1. Also, these transformations do not make any sense at all with categorical variables. For example, to standardize a column, you have to substract the mean from each value and divide it by the standard deviation of the column. Assume, you try to standardize the categorical column "animals" that holds values like "cat", "dog", "horse", etc. I hope it is clear, that such a column cannot have a mean or a standard deviation. $\endgroup$ – georg-un Jul 23 at 7:52
0
$\begingroup$

I mean after encoding the categorical variables using Target encoding, can we use mean normalization? And would it be useful?

I think it depends. A priori I'm inclined to think this doesn't make any sense, but let's think about it. You are encoding each value with a number which is somewhat related to the target. In what range do these numbers live? Now, I think the main way to answer the question is by inspecting the following question:

What is the purpose of normalization?

There are algorithms which can understand greater values have greater meaning, and by simply having features with different ranges prevents the algorithm to perform well. Here it seems like it would make sense to normalize.

Here, Urvashi Jaitley explains that without normalization,

gradients may end up taking a long time and can oscillate back and forth and take a long time before it can finally find its way to the global/local minimum. To overcome the model learning problem, we normalize the data.

After thinking a bit about this, although it didn't make much sense to me at the beginning, I now think normalization would apply in this case too.

As of the comment on @georg_un's answer:

If we one-hot or target encode the categorical variables, will standardization or normalization help the model? Or I can simply not apply any sort of scaling and algorithm works fine anyway?

If you one hot encode the variables, the values will be either 0 or 1. Here it really doesn't make sense to normalize. What you want is a binary variable to indicate something as either yes or no. You don't want grays there.

Hope this helps!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.