Does one-hot encoding result in information loss?

Question

BACKGROUND: I'm working with a nominal feature variable cancer_type with $5$ different classes to develop a machine learning model.

One-hot encoding this feature column will result in $5$ columns, one for each class, in the new data frame. The information that these columns are linked is lost with this procedure because the model has no way of knowing that they refer to the same original feature. The columns are in a sense independent from the perspective of the model, whereas we know that there is a dependency. (A $1$ in any one of the one-hot encoded columns necessitates that all other columns must have a $0$ for that particular training example.)

QUESTION: Does one-hot encoding result in information loss?

Answer 1

If your nominal variable is truly nominal then there is no loss of information if you split up this variable into multiple variables, you have to in order to use a model which only uses numerical variables anyway. In this case the categories are completely distinct and independent, say cats and dogs, one category has no relation to the other, the variable creates two distinct partitions of the data.

However, if you variable is ordinal, let's say your cancer variable is a scale variable from 1 to 5 which describes the degree of cancer presence, then splitting up this variable does lose information, since the model has no way of knowing that 2 follows from 1 and that this is describing the same underlying event but with different degrees. In such cases you do not want to one-hot encode, maybe.

Answer 2

There is no information loss because one-hot-encoding has a 1-to-1 correspondence so you can invert the process without a problem.

In order to get back the original data, you can apply this code:

s = pd.Series(['dog', 'cat', 'dog', 'bird', 'fox', 'dog'])

In [41]: s
Out[41]:
0     dog
1     cat
2     dog
3    bird
4     fox
5     dog
dtype: object

In [42]: pd.get_dummies(s)
Out[42]:
   bird  cat  dog  fox
0   0.0  0.0  1.0  0.0
1   0.0  1.0  0.0  0.0
2   0.0  0.0  1.0  0.0
3   1.0  0.0  0.0  0.0
4   0.0  0.0  0.0  1.0
5   0.0  0.0  1.0  0.0

In [43]: pd.get_dummies(s).idxmax(1)
Out[43]:
0     dog
1     cat
2     dog
3    bird
4     fox
5     dog
dtype: object

Source: https://stackoverflow.com/questions/38334296/reversing-one-hot-encoding-in-pandas