12
$\begingroup$

I am reading a presentation and it recommends not using leave one out encoding, but it is okay with one hot encoding. I thought they both were the same. Can anyone describe what the differences between them are?

$\endgroup$
  • 1
    $\begingroup$ It's not clear (from just your question) what leave-on-out even is. You should edit this to give a pointer and explain briefly your understanding of the two, and why you think they are the same. $\endgroup$ – Sean Owen Mar 23 '16 at 13:31
  • $\begingroup$ leave one out, from scikit learn contrib categorical project $\endgroup$ – mork Mar 18 at 8:34
14
$\begingroup$

They are probably using "leave one out encoding" to refer to Owen Zhang's strategy.

From: https://www.kaggle.com/c/caterpillar-tube-pricing/forums/t/15748/strategies-to-encode-categorical-variables-with-many-categories

The encoded column is not a conventional dummy variable, but instead is the mean response over all rows for this categorical level, excluding the row itself. This gives you the advantage of having a one-column representation of the categorical while avoiding direct response leakage

This picture expresses the idea well. enter image description here

$\endgroup$
  • $\begingroup$ Your explanation is better than wacax's in the referred link, thank you $\endgroup$ – Allan Ruin Aug 12 '16 at 15:00
  • $\begingroup$ Hi @Dex Groves, so the leave_one_out encoding for the test is always .5? $\endgroup$ – user7117436 Mar 24 '17 at 20:29
  • 3
    $\begingroup$ Hi! As seen from the picture, this paticular example relates to classification problem. Does anybody have an experience with LOO encoding within regression problem? The main question is how to aggregate the target variable. I am now making experiments and get huge overfitting with mean(y). $\endgroup$ – Alexey Trofimov Jun 19 '17 at 12:49
  • 1
    $\begingroup$ for a clustering (unsupervised) problem, is possible to use this kind of encoding? $\endgroup$ – enneppi Sep 13 '18 at 10:26
  • $\begingroup$ @AlexeyTrofimov - try an aggregation with a lower variance. I'd start with different binning (like 1K, 2K, 2M, .. for large y int values, or some rounding to a decimal place for y float values) => mean(bin_f(y)) $\endgroup$ – mork Mar 18 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.