I am exploring the possibility of data leakage for categorical columns, replacing rare values with category 'Other'
Let's say I have a DF with 40 categorical columns. I will check each of them, find frequencies of unique values, and If a value is less frequent than 1%, I will replace it with category value of 'Other.
Same process will be applied to each column, most probably with a loop.
Actually I am highly suspicious of data leakage, when applying this directly to whole data. In the process, I will apply a CV scheme. And I am also considering to put it into preprocessing pipeline, calculate rare categories only with Train folds, each time. While training the production model at the end, I can use whole data of course.
I want to ask, Is it overkill to put this replacement into CV pipeline, and apply several times for only Training data? (Since I have 40 categorical columns, it may mean 40*N times of calculation, for N fold CV)
If I apply it to whole data at the beginning, will I be prone to data leakage?
Thanks.