# What is the difference between one-hot and dummy encoding?

I am trying to understand

• The reason behind encoding (one-hot encoding and dummy encoding)
• How one-hot and dummy are different from each other
• See this for an example and my answer, so you best understand what OneHot does and how it works : datascience.stackexchange.com/q/80809/101580 Jul 22, 2021 at 7:10
• One-hot encoding and dummy encoding historically mean the exact same thing. The former term originated from machine learning, the latter from statistics. However, it does seem that over the years the two have separated to represent whether to drop one level as in Archana's answer. Just be aware that not everyone will take that view (e.g. the first comment on this question). Jul 29, 2021 at 18:50

Most machine learning models accept only numerical variables. This is the reason behind why categorical variables are converted to number so the model can understand better.

Now lets address your second query lets look into what is one-hot encoding and dummy encoding and then see the difference

• One hot Encoding: Take the example of column name Fruit which can have different types of fruits like Blackberry, Grape, Orange. Here each category is mapped to binary variable containing either 0 or 1. Widely utilized when features are nominal.

Fruit Price (dollars per pound)
Blackberry 3.82
Grape 1.2
Orange .64

Post one hot encoding the table now looks as shown below

###### One Hot Encoded table
Blackberry Grape Orange Price (dollars per pound)
1 0 0 3.82
0 1 0 1.2
0 0 1 .64
• Dummy Encoding: similar to one hot encoding. While one hot encoding utilises N binary variables for N categories in a variable. Dummy encoding uses N-1 features to represent N labels/categories

##### One Hot Coding Vs Dummy Coding
Column One Hot Code Dummy Code
Blackberry 100 10
Grape 010 01
Orange 001 00

From what I encountered, the big advantage of sklearn.preprocessing.OneHotEncoder is that you can save it as an scikit-learn encoder, so you can train it on a train set, and apply it on your test based on what you train (you'll recreate the same columns). On the other hand, pandas.get_dummies only applies directly on your dataframe, so you won't be able to train it on a set and then apply it to another based on what you trained first. This then causes issues with machine learning problems. Basically always use one-hot.

Example:

Training set:

Dog
Cat
Rabbit


Train your one-hot encoder on that and apply it:

Dog  Cat  Rabbit
1    0    0
0    1    0
0    0    1


Test Set:

Dog
Horse
Cat


The one-hot encoder trained from your training set applied to your test:

Dog  Cat  Rabbit
1    0    0
0    0    0
0    1    0


If you used Pandas pandas.get_dummies, you could only apply it directly on your test:

Dog  Horse Cat
1    0     0
0    1     0
0    0     1


And you'll have a column mismatch between your train and test: ['Dog', 'Cat', 'Rabbit'] differs from ['Dog', 'Horse', 'Cat']

• Note that sklearn's OneHotEncoder has a parameter drop to allow for dropping reference levels. Jul 29, 2021 at 18:51

The purpose of one-hot encoding is to assign numbers to categorical variables which does not create a false, meaningless numerical pattern.

If you have categorical variables "Apple", "Orange", "Cherry", "Tomato" and you assign them numerical values 0, 1, 2, 3, then these numerical values have interpretations like "Cherry is between Tomato and Apple, but closer to Tomato" because 2 is between 0 and 3, but closer to 3. This is nonsense. It's bad nonsense, because algorithms to analyze this data (like regressions, or whatever) can pick up on it and read too much into it.

If you instead represent "Apple", "Orange", "Cherry", and "Tomato" as the 4-tuples (1,0,0,0), (0,1,0,0), (0,0,1,0), and (0,0,0,1), then you don't have this problem. Each coordinate is either 0 or 1, and measures the "Appleness" or the "Orangeness" or the "Cherriness" or the "Tomatoness" of your fruit. That's one-hot encoding.

As an example of this, suppose that the average apple weighs 200 grams, the average orange weighs 150 grams, the average cherry 30 grams, and the average tomato 100 grams. With one-hot encoding $$(x_1, x_2, x_3, x_4)$$, this average weight is a linear function of the encoding: $$200x_1 + 150x_2 + 30x_3 + 100x_4$$. This is something a regression can figure out from data. With the 0, 1, 2, 3 encoding, there's no nice function that will give you the average weight of a fruit given its number.

Now, as for dummy encoding: one-hot encoding still has a problem, which is that the linear function is not unique. The function $$100 + 100x_1 + 50x_2 - 70x_3$$ gives the same values as the previous function at the points (1,0,0,0), (0,1,0,0), (0,0,1,0), and (0,0,0,1). That's because each valid point $$(x_1, x_2, x_3, x_4)$$ satisfies $$x_1 + x_2 + x_3 + x_4 = 1$$.

(Again, this is not just a curiosity; this affects the way we analyze the data. For example, linear regressions behave badly when an $$n$$-dimensional input doesn't actually range freely across all $$n$$ dimensions.)

Dummy encoding drops one of the coordinates, since it can be inferred from the other three, to avoid this issue. The four fruits might be encoded as (1,0,0), (0,1,0), (0,0,1), and (0,0,0).

• Your last example doesn't seem to hold? For (1,0,0,0) it's 200, for (0,1,0,0) it's 150, etc. Jul 23, 2021 at 8:57
• Aren't those the values it's supposed to have? e.g., (1,0,0,0) is an apple, which weighs 200 grams, and both functions give 200 and (1,0,0,0). Jul 23, 2021 at 13:24
• Ah, ok. I misunderstood. So you meant it gives the same answer as the previous linear function. I thought it gives the same answer for all four points. Jul 23, 2021 at 19:27