I am learning about label encoders, one hot encoding etc applied to datasets for classification via KNN and XGBoost type trees. However, I am a bit confused as to whether the target variable should be one hot encoded or not.

In my case, I have a dataframe which has 500x10 size and the last column is the target (for prediction) country column while the rest of the 499 columns contain floating point values. So, an example row:

feature1 ..... fetaure499 target
1.1  ......... 40.1       Germany
6.1  ......... 265.7       USA
8.1  ......... 98,7      Singapore

In this case, what is the optimal way to encode the target for: [1] KNN [2] tree based methods.

My relatively naive understanding says that one hot encoding is useful for the feature space while for the target such as this, i.e list of countries labelencoding should be enough. Is this true for this case? Are there any situations I need to aware of when the target has to be label encoded?

Thank you.

  • 2
    $\begingroup$ Does this answer your question? Encode multi-class response variable $\endgroup$ – Ben Reiniger Apr 19 at 14:56
  • $\begingroup$ @BenReiniger yea, it kinda does - so, the essense is that scikit-learn does not care about the target output i.e one can labelencode the target output and thats it. Is that correct? Does this apply to say xg-boost aswell? $\endgroup$ – AJW Apr 19 at 16:49
  • $\begingroup$ I'm not sure offhand how xgboost handles its targets. For sklearn, I'd suggest not encoding at all: that way it still knows what the targets are, often exposed through the classes_ attribute. $\endgroup$ – Ben Reiniger Apr 20 at 14:24

The answer depends on the library you are using. If you are using scikit - you wouldn't need to one hot encode the targets. Scikit handles it automatically.

If you were using keras to build a neural network, you might want to use one hot encoded labels because the built in loss function in keras (e.g categorical crossentropy) expects labels to be one hot encoded.

So both approaches work and ate a function of the library or tools you use.

  • $\begingroup$ this is irrelevant to the question. It does not ask for software tools but a theoretical question $\endgroup$ – Nikos M. Apr 19 at 16:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.