I am making a classification model. I have categorical and continuous data. The categorical columns include columns with 2 classes such as sex (male, female), and multi-class columns such as location.

I need to encode these to numeric values. I would do one-hot-encoding and drop first column but it is not realistic on an unseen test data that may have unseen values. so I have planned to do one-hot-encoding with handle_unknown='ignore' .

However, my problem is that I am afraid of the multicollinearity this presents in the data, especially for the columns with 2 classes.

The solution I have thought of is only applying LabelEncoder on the columns with 2 classes, and one-hot-encoder for the rest. This way the effects of multicollinearity is lessened.

does that seem right?

Please let me know what you think. Thank you.

  • $\begingroup$ OneHotEncoding or LabelEncosing are not the only 2 techniques you can use. There are many other techniques which you can choose depending upon the type of categorical variable you have.\ $\endgroup$
    – spectre
    Jan 21, 2022 at 5:42

1 Answer 1


You do not need to do one-hot encoding for a variable with 2 categories. You simply need to index as 1 and 0.

As a matter of fact, if you did do one hot encoding for a binary variable then you would end up with a multi-collinearity as you said, as those variables would be exactly the same.

The same thing occurs with any other categorical variable. If you have a variable with 3 categories, and you do one hot encoding, all the information you need is carried in two of the one hot encoded variables. Therefore, you should also do drop first for any categorical variable. I am not sure what you mean by unseen data or why that would matter.

  • $\begingroup$ The reason why I can not do drop first is when I apply the encoder to test data and there are unseen values in the test data that the encoder has not seen, it does not work. $\endgroup$
    – Alis
    Jan 21, 2022 at 12:41
  • $\begingroup$ Oh that is interesting problem. how does this arise? $\endgroup$ Jan 21, 2022 at 14:39
  • $\begingroup$ The problem arise when unknown/unseen value is encountered by the encoder during transform on the test data. You would usually get around this by setting handle_unknown to ignore: OneHotEncoder(handle_unknown='ignore'). This means we can not do drop one because it is not allowed when we are using handle_unknown. $\endgroup$
    – Alis
    Feb 1, 2022 at 13:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.