How do I perform frequency/count encoding for a train and test set?
The implementations of this encoding I've seen simply frequency encode the categorical variables on a particular dataset (no specific train, and test encoding transformation). For instance:
In my case now I have a train, and test set.
[First option] Is it okay (due to leakage? or there won't?) for me to use frequency encoding on the whole dataset. OR
[Second option] I should take into consideration train, and test set independence.
If the second option, how do I do this?
- Encode the train set, then use the encoding values of categories in the train set for the same categories in the test set. Categories not represented in the test set would be need to be handled. OR
- There's a better generic implementation?