- is it true that Label Encoding will be misinterpreted as a numeric scale by scikit-learn trees?
Not mandatory for a TreeYes, SciKit-Learn treats it as Numeric value.
A Tree simply slices the space into multiple parts using multiple if-else(in a very simple language) on different features and values. It learns the bestHence, if-elseit will impact the depth of Tree and result in different Tree structure using the training data.
SoOn results - Definitely, itdifferent hyperparameter tuning will be required for different methods but I am not mind whatsure about the fact that whether we will never achieve the best with Label encoding Or we may if tuned properly.
It is also true that if the encoding is aligned with Labels/target, even if you don't encode(Library limitation/efficiency is a separate thing)it will achieve a good result quickly.
May like to read this Answer
- if so, are there any situations at all where arbitrary Label Encoding can be useful or does this technique has no use at all unless the variable is ordinal, and a specific labeling order is given (i.e. Ordinal Encoding is useful only when it's truly ordinal)?
I doubt that it will work i.e. with Neural Network Or Linear Regression, etc.
10 will become 2 times of 5 without any such underlying relation between two values of a Feature.
If it happens, it will be a coincidence or might be because of a subconscious knowledge about the Target(Target encoding) while assigning the value randomly.
but now I suspect that that whole lesson is best removed altogether to avoid teaching students bad practices
I don't think that it's a bad practice or an anti-pattern.
It just that you I think students should know the logic behind it and usehow it as per the casewill fail/behave in different conditions. Ordinal encoding definitely has its usefulness based onSo that they can grasp the case i.e. when the Values follow an Order and the difference between each is sameunderlying concept.