# Frequency/Count encoding

How do I perform frequency/count encoding for a train and test set?

The implementations of this encoding I've seen simply frequency encode the categorical variables on a particular dataset (no specific train, and test encoding transformation). For instance:

dataset.groupby("cat_column").size()/len(dataset)



In my case now I have a train, and test set.

[First option] Is it okay (due to leakage? or there won't?) for me to use frequency encoding on the whole dataset. OR

[Second option] I should take into consideration train, and test set independence.

If the second option, how do I do this?

1. Encode the train set, then use the encoding values of categories in the train set for the same categories in the test set. Categories not represented in the test set would be need to be handled. OR
2. There's a better generic implementation?

I hope this is not a late answer but actually you can use category_encoders library, it follows sklearn's style.

Example:

import category_encoders as ce