2
$\begingroup$

This question is from this post asking about what is the difference between one-hot encoding and leave-one-out encoding.

I now understand how to calculate the leave-one-out encoding for the training set. But how about the test set, where is Y (label) is blinded? If the encoding for test set is always 0.5, then what is the point of this encoding?

$\endgroup$
1
$\begingroup$

The point is to learn useful variations of data instead of just splitting by large categorial variable. Each new row after encoding becomes immediately related with the output, while original categorial variable may be related only in indirect, latent manner. Plus, the interactions between output and the original variable are represented too, by definition. Think of it as of adding this interaction explicitly to add more intuitive justification to this method.

So I think this method will be perfect not only for divide-and-conquer RF-style methods but also for plain LR as well.

if the encoding for test set is always 0.5, then what is the point of this encoding?

That's okay, as soon as you know how this should be related to the output. 0.5 in this example is just a rough approximation of input, but given that you have learned model even this approximation is still meaningful, as it instructs model -- which path to output to choose. If your model of choice is RF then 0.5 will land at about same set of leaves as more granular distribution ( if you were able to know it )

Meanwhile much stronger approximations are possible, for instance, one can use KDE to estimate response variable distribution and then draw samples from it at test time. Adding uniform random noise is just a hint in this direction. Maybe you want to follow it?

Knowing only particular level of categorial variable is way less informative, than this 0.5, isn't it?

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.