We know that we prefer to using One-Hot Encoding not Label Encoding when processing with non-ordinal data.
And I real a blog which give the difference between Label Encoding and One-Hot Encoding.
So I am wondering why One-Hot Encoder can avoid the situation that the model will misunderstand the data to be in some kind of order, $0 < 1 < 2$ if the data has been Label Encoding.
Does it have some provements and theories?
Or anyone just can explain it in intuition.
Thanks in advance.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ To be honest I haven't seen any difference $\endgroup$– WoofDoggyCommented Apr 25, 2019 at 15:05
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
From the blog, one hot encoding resolves the issue by explicitly showing that 1 category is true, while all others are false (1 vs 0). One column is turned into 3 columns which describe all categories as either true or false.
As opposed to letting a model see one column with 0, 1, and 2. One column to show this data with label encoding does make it seem as if the data is numerical so 0 < 1 < 2.
$\begingroup$
$\endgroup$
Good question!
This might help:
Let's say you have three countries: USA, Germany, and China. No ranking.
Label Encoding turns the countries into numbers.
For example, 1 (USA),2 (Germany), and 3 (China).
If you don't One Hot Encode, the theory states (source: Sebastian Raschka "Machine Learning with Python"), that you create ordinality.
In other words, USA is better than Germany and Germany is better than China.
As you have no ranking (order), you don't want that. Thus, we One Hot Encode.
Personally, I have tested this several times and only Label Encoded (not ranking features) with Logistic Regression and have NOT seen a difference. But I've only done it a few times. And, if an ML expert like Sebastian Raschka says we should One Hot Encode for non-ordinal numerical data than I trust him.