Skip to main content
Edited as per the comment
Source Link
10xAI
  • 5.8k
  • 2
  • 9
  • 25
  1. is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?

Not mandatory for a TreeYes, SciKit-Learn treats it as Numeric value.
A Tree simply slices the space into multiple parts using multiple if-else(in a very simple language) on different features and values. It learns the bestHence, if-elseit will impact the depth of Tree and result in different Tree structure using the training data.
SoOn results - Definitely, itdifferent hyperparameter tuning will be required for different methods but I am not mind whatsure about the fact that whether we will never achieve the best with Label encoding Or we may if tuned properly.
It is also true that if the encoding is aligned with Labels/target, even if you don't encode(Library limitation/efficiency is a separate thing)it will achieve a good result quickly.
May like to read this Answer

  1. if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?

I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.

but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices

I don't think that it's a bad practice or an anti-pattern.
It just that you I think students should know the logic behind it and usehow it as per the casewill fail/behave in different conditions. Ordinal encoding definitely has its usefulness based onSo that they can grasp the case i.e. when the Values follow an Order and the difference between each is sameunderlying concept.

  1. is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?

Not mandatory for a Tree.
A Tree simply slices the space into multiple parts using multiple if-else(in a very simple language) on different features and values. It learns the best if-else using the training data.
So, it will not mind what the encoding is, even if you don't encode(Library limitation/efficiency is a separate thing)
May read this Answer

  1. if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?

I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.

but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices

I don't think that it's a bad practice or an anti-pattern.
It just that you should know the logic behind it and use it as per the case. Ordinal encoding definitely has its usefulness based on the case i.e. when the Values follow an Order and the difference between each is same.

  1. is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?

Yes, SciKit-Learn treats it as Numeric value.
Hence, it will impact the depth of Tree and result in different Tree structure.
On results - Definitely, different hyperparameter tuning will be required for different methods but I am not sure about the fact that whether we will never achieve the best with Label encoding Or we may if tuned properly.
It is also true that if the encoding is aligned with Labels/target, it will achieve a good result quickly.
May like to read this Answer

  1. if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?

I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.

but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices

I think students should know how it will fail/behave in different conditions. So that they can grasp the underlying concept.

Source Link
10xAI
  • 5.8k
  • 2
  • 9
  • 25

  1. is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?

Not mandatory for a Tree.
A Tree simply slices the space into multiple parts using multiple if-else(in a very simple language) on different features and values. It learns the best if-else using the training data.
So, it will not mind what the encoding is, even if you don't encode(Library limitation/efficiency is a separate thing)
May read this Answer

  1. if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?

I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.

but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices

I don't think that it's a bad practice or an anti-pattern.
It just that you should know the logic behind it and use it as per the case. Ordinal encoding definitely has its usefulness based on the case i.e. when the Values follow an Order and the difference between each is same.