0
$\begingroup$

I'm trying to do a simple softmax regression where I have features (2 columns) and a one hot encoded vector of labels (two categories: left = 1 and Right = 0). Do I need to standardize just the vector of features or also the vector of labels? when I do that all my zeros and ones transform in different numbers and also I don't know how to identify who is the Left or the Right category. I'm using mxnet and gluon. Here is how I standardize: labels = (labels - labels.mean()) / (labels.max() - labels.min())

labels before standardization: [0. 1. 1. 1. 1. 1.

labels after standardization: [-0.5633803 0.43661973 0.43661973 0.43661973 0.43661973 0.43661973 ...

How can I after identify (with strings) if my prediction is actually giving me Left or Right?

$\endgroup$
1
$\begingroup$

NO, you do not standardize labels

The purpose of standardization is to bring features with disparate ranges into a standard range. When the data is not standardized, features with large numerical values will tend to have a larger influence (weight) than those that are smaller numerically.

Consider the Automobile Data Set (https://archive.ics.uci.edu/ml/datasets/automobile) from UC Irvine. Among the features, it has height, lenght, width, weight, # cylinders, along with a number of other label and numerical features. Height ranges from from 47.8 to 59.8 and weight ranges from 1488 to 4066. You'll want to standardize them, but not label features such as body-style or engine-type.

$\endgroup$
0
$\begingroup$

From what I can tell, there isn't a "right" answer to the title question. Most people I know wouldn't bother. (Indeed, one often-used scaler puts the data into the range $[0,1]$ anyway.) https://stats.stackexchange.com/questions/290929/standardizing-dummy-variables-for-variable-importance-in-glmnet
https://stats.stackexchange.com/questions/359015/ridge-lasso-standardization-of-dummy-indicators

For the second question, normalization/standardization is always (er, always that I've seen, and certainly for your example) applying an increasing function, so that the ordering is preserved. For your normalization of binary variables then, always 0 gets mapped to the negative value and 1 to the positive one.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.