From what I read online, there seems to be some confusion regarding the taxonomy and the terms used, so to avoid misunderstanding I'm going to define them here:

Label Encoding - encoding a nominal variable with arbitrary numeric labels.

Ordinal Encoding - encoding an ordinal variable with numeric labels arranged in a specific order.

The course on Machine Learning I'm currently taking compares One-Hot Encoding with Ordinal Encoding. However, during my research online I came to realize that "Ordinal Encoding" is actually a misnomer, and what that course actually demonstrates is called "Label Encoding". Ordinal Encoding is supposed to pertain strictly to ordinal variables, and the dataset in question didn't even have any ordinal variables.

Where did that misnomer come from? Turns out that it comes from the scikit-learn library that has LabelEncoder and OrdinalEncoder classes. The thing is, OrdinalEncoder class actually does not perform Ordinal Encoding by default. To make it ordinal, you have to specify the order in the 'categories' parameter (and its usage is extremely not-user-friendly - dictionary mapping by pandas can do the same way easier). If you don't, OrdinalEncoder will assign labels alphabetically, just like LabelEncoder does. So the real difference between these two classes is that one encodes only a single column, while the other encodes two or more columns at a time. Perhaps it would be better and much less confusing if these classes were called "LabelEncoder1D" and "LabelEncoder2D".

So that's where mistakenly calling Label Encoding "Ordinal Encoding" is coming from. But getting back to the question, the course I'm taking advocates the usage of (what I learned to be) Label Encoding for tree-based algorithms, because One-Hot encoding performs much worse for trees (which is ture). However, from what I read online, it appears that other Machine Learning platforms, such as R or H2O, are capable of processing nominal variables for trees without any kind of encoding at all, and the requirement to encode everything into numeric form seems to be exclusively scikit-learn's problem. And there's conflicting information as to whether trees perform better with Label Encoding - my course, as well as some responses online, advocate for its usage, but my intuition, as well as some other responses online, seem to indicate that scikit-learn trees will not be able to distinguish these labels as categories, and will mistakenly assume that they are continuous values on a meaningful scale instead. So they recommend using One-Hot Encoding even for trees as the only option despite it being sub-optimal.

So my questions are 1) is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees? 2) if so, are there any situations at all where arbitrary Label Encoding can be useful? Or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given?

P.S: I'm asking because my course has a whole lesson dedicated to teaching students "Ordinal" Encoding. At first I wanted to suggest them to rename it to "Label Encoding", but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices.


2 Answers 2


First, I generally agree that encoding unordered categories as consecutive integers is not a great approach: you are adding a ton of additional relationships that aren't present in the data.


First, let me point out (because I nearly forgot) that there are two main types of decision tree: CART and the Quinlan family. For the Quinlan family, categorical variables are dealt with by using higher arity splits, so no encoding is needed and this is mostly moot.

Q1, yes, ordinally encoding will be treated by the model as numeric (unless some other parameter controls that, e.g. LightGBM). But for (most) trees, only the order is actually relevant: the scale is irrelevant, and e.g. the relationship "10 is twice as much as 5" is completely invisible to the tree.

As you point out, one-hot encoding for a CART model can be detrimental, especially when there are many levels in a categorical feature: will the tree ever actually decide to split on one of the dummy variables, if it is only 1 for a small subset of the data? (Q2)But when you encode ordinally, there will just by chance be some splits that are useful and split many levels in each direction. (You may even try more than one random ordering of the levels as different features!)

And yes, presumably the best approach is to use an implementation that can take advantage of the raw categoricals, using the average-response trick. (There's even some debate on how much that helps: some studies have been done, but generally the datasets are synthetic or too small to be representative.)

Other models

In other models, very often one-hot encoding is just fine, and doesn't suffer from the same problem as trees. If there are too many levels, and especially if some of them are too small, you may consider smoothing techniques to avoid overfitting. (Q2)I'd be surprised if ordinally encoding is ever worth it for most models, but one would need to consider each model type individually, and probably do some testing.


As for naming, things are a bit muddy, but I don't think this is sklearn's fault. The "label" in LabelEncoder means it is supposed to be used on the labels, a.k.a. the dependent variable. And for that usage, there is no debate about whether it's appropriate: sklearn just requires consecutive integer labels for its multiclass classification; it doesn't use the numeric values as though they were mathematically meaningful.

As for OrdinalEncoder, it is meant to be used with input ordering of the categories. See sklearn Issue#13488 below. But one could argue that you are encoding the categorical variable in an ordinal way, so even with unordered categories this isn't necessarily a misnomer.

See also

See Issue#13488 for some related discussion.

  • $\begingroup$ Thanks, this is helpful. And separate thanks for the link to the relevant GitHub issues. Judging by Issue#14653 the sklearn team did consider limiting the usage of OrdinalEncoder strictly to ordinal variables, but in the end they decided to keep both kinds of encoding in the same class, though they still intend to deprecate automatically assuming lexicographic ordering version 0.24 onward. $\endgroup$ Jul 21, 2020 at 2:41
  • $\begingroup$ And this comment implies that at least one of the developers considers it acceptable to use arbitrary labels for tree-based algorithms. Though I'm still skeptical and would much prefer the raw categorical support. As far as I understand, they are working on that too. $\endgroup$ Jul 21, 2020 at 2:45
  • $\begingroup$ Also, while the developers intended LabelEncoder to be used on the dependent variable, in practice many people seem to use it on a regular feature too, because currently this is the only straightforward way to encode a single feature. OrdinalEncoder currently requires to pass it at least two columns, which is often inconvenient. $\endgroup$ Jul 21, 2020 at 2:54
  1. is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?

Yes, SciKit-Learn treats it as Numeric value.
Hence, it will impact the depth of Tree and result in different Tree structure.
On results - Definitely, different hyperparameter tuning will be required for different methods but I am not sure about the fact that whether we will never achieve the best with Label encoding Or we may if tuned properly.
It is also true that if the encoding is aligned with Labels/target, it will achieve a good result quickly.
May like to read this Answer

  1. if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?

I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.

but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices

I think students should know how it will fail/behave in different conditions. So that they can grasp the underlying concept.

  • $\begingroup$ I understand that decision thees as a whole can support categorical variables. The question was specifically about scikit-learn implementation. I also found this question, and it seems to indicate that scikit-learn trees misinterpret Label Encoding to be a continuous numeric variable. Your link also says that sklearn does not support categorical variables. $\endgroup$ Jul 17, 2020 at 18:29
  • $\begingroup$ And I also understand that Ordinal Encoding has its uses. The question was specifically about Label Encoding. Label Encoding - encoding a nominal variable with arbitrary numeric labels. Nominal variables don't follow an order, that's why they are called nominal. Is there ever a use for arbitrary numeric labels, not ordered? $\endgroup$ Jul 17, 2020 at 18:29
  • $\begingroup$ Updated, though not very conclusive i.e. Black & White. I missed a few of your points and jumped to the end of the question. $\endgroup$
    – 10xAI
    Jul 18, 2020 at 7:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.