0
$\begingroup$

In the context of algorithms that consider the scales of features, I have a situation where some features are encoded using ordinal encoding, some features are binary, and some features are standard scaled. This results in three different ranges of distribution: 1-5 for ordinal encoded features, 1 or 0 for binary features, and approximately -1 to 1 for standardized numerical features. My question is whether all encoded categorical features, including binary ones, should be standardized?

$\endgroup$
1
  • $\begingroup$ If the model (optimization) depends on it, then yes. $\endgroup$ Jun 27, 2023 at 11:47

2 Answers 2

1
$\begingroup$

To answer your question:

  • Standardizing features is generally a good practice when using machine learning algorithms that are sensitive to the scale of the features, such as Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), and Neural Networks.

  • However, whether to standardize binary and ordinal features is not a straightforward decision and depends on the specific context and the algorithm you are using.

  • For binary features, standardization will transform them into continuous variables, which might not be desirable, especially if they represent distinct categories. For example, if you have a binary feature representing "male" and "female", standardizing it might not make sense as it could lose its interpretability.

  • For ordinal features, standardization could potentially distort the relative distances between the categories. For example, if you have an ordinal feature representing "low", "medium", and "high", standardizing it could change the relative distances between these categories.

  • In general, it's a good idea to experiment with different approaches and see which one works best for your specific problem. You could try standardizing all features, only numerical features, or no features at all, and see which approach gives you the best results.

Hope I answered your question.

$\endgroup$
0
$\begingroup$

those three types must be different outputs. means 3 different prediction or output nodes. no need to scale output variable in this case it will be handled automatically.

scaling target variable very rarely needed: https://stats.stackexchange.com/questions/111467/is-it-necessary-to-scale-the-target-value-in-addition-to-scaling-features-for-re

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.