14
$\begingroup$

Frequency encoding is a widely used technique in Kaggle competitions, and many times proves to be a very reasonable way of dealing with categorical features with high cardinality. I really don't understand why it works.

Does it work in very specific cases where frequencies are correlated with the target or is it more general? What's the rationale behind it?

$\endgroup$
2
  • $\begingroup$ maybe this answer $\endgroup$
    – Nikos M.
    Jul 26, 2020 at 8:26
  • $\begingroup$ @NikosM. Can you copy that answer here? $\endgroup$ Jul 30, 2020 at 7:14

2 Answers 2

8
$\begingroup$

Check this post.

In the cases where the frequency is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on the nature of the data.

Check also this thread.

What's the rationale behind it?

High cardinality may result in dimensionality curse and actually decrease the quality of your model.

$\endgroup$
2
  • 1
    $\begingroup$ The second thread talks about something different I think. I understand it is talking about target encdoing. And about the dimensionality curse, target encoding also helps solving it but frequency encoding is preferred sometimes. Why is that? $\endgroup$ Nov 25, 2019 at 16:13
  • 1
    $\begingroup$ Frequency may convey some actual domain knowledge. Stating it explicitly may help and keeps dimensionality low. All of this is a matter of debate and I wouldn't blindly follow it. $\endgroup$ Nov 25, 2019 at 16:15
1
$\begingroup$

Frequency Encoding


It is a way to utilize the frequency of the categories as labels. In the cases where the frequency is related somewhat with the target variable, it helps the model to understand and assign the weight in direct and inverse proportion, depending on the nature of the data. Replace the categories with the count of the observations that show that category in the dataset. Similarly, we can replace the category by the frequency -or percentage- of observations in the dataset.

It can help if frequency correlates with the target and also, it can help the model to understand that smaller categories are less trustworthy than bigger ones, especially when frequency encoding is used parallel with other types of encoding.

Advantages of Count or Frequency encoding

  • Straightforward to implement.
  • Does not expand the feature space.
  • Can work well with tree-based algorithms.

Limitations of Count or Frequency encoding

  • Does not handle new categories in the test set automatically.
  • We can lose valuable information if there are two different categories with the same amount of observations count—this is because we replace them with the same number.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.