Basically the title.
If I encode the address of people (the cities they live in) with a target encoder, do I still need to normalize that column? Of course, the capital is going to have more citizens and bigger cities also, so it looks kinda like an exponential distribution. In such a case, is normalization still needed (via a log transform for example), or are target encoded variables enough? Why?
Thank you!
(I can't leave comments yet so I'll have to write an answer instead.) It depends on what kind of model you're using. And more specifically...
If you're doing using something like logistic regression, or you're using neural networks with non-linear activations (ReLU, tanh, sigmoid, softmax especially), you absolutely need to do some type of normalization/standardization on all the features you decide to incorporate because the loss functions and their gradients are susceptible to unfriendly behavior.
Another class of models you would need to normalize/standardize data in are in clustering models, because those rely on the notion of a metric for the partitioning criteria. If you are incorporating your categorical variables into a clustering algorithm you had better normalize them.
On the flipside, if you're doing something like random forest or decision trees (also applies to gradient boosted analogues), it's not necessary to normalize your data because the partitioning criteria in those models is almost always independent of the scale of the data.
So the answer is, it depends on the model, and it depends on how the model partitions the inputs to achieve the output (using a loss function, discrete conditions to send down branches of a tree, etc.)
Normalization is not typically necessary for target encoded variables. However, whether you should normalize depends on the specific machine learning algorithm you are using. Some algorithms, like linear regression or k-nearest neighbors, can be sensitive to the scale of the input features, and normalization can help in these cases. Other algorithms, like tree-based methods, are not sensitive to the scale of the features, so normalization is not necessary.
In general, it's a good practice to try both with and without normalization and see which gives better performance on your validation set.
