# Possible harm in standardizing one-hot encoded features

While there may not be any added value in standardizing one-hot encoded features prior to applying linear models, is there is any harm in doing so (i.e., affecting model performance)?

Standardizing definition: applying (x - mean) / std to make the feature mean and std 0, 1 respectively)

I prefer applying standardization to my entire training dataset after one-hot encoding, rather than applying it only to the numerical features. I feel it would significantly simplify my pipeline.

For example, if I have a binary feature then the vector that will be provided to the model is [1,1,0,0,0,1,1].

If standardization is applied to this binary feature prior to fitting the model (subtract mean = ~0.57 and divide by std = ~ 0.49), the vector will become

[ 0.8660254 , 0.8660254 , -1.15470054, -1.15470054, -1.15470054, 0.8660254 , 0.8660254 ]

• What linear model? OLS? And what language? If you use R, for instance, and you standardize, and AFTER that apply as.factor(), you are fine. Otherwise, when you do not apply a „as.factor“ function but feed raw numbers into the regression, „dummies“ should be either 1 or 0. Commented Aug 13, 2020 at 21:37
• The question is unrelated to a specific language and it is more about the theory of standardizing features which are binary. By Linear models I mean Linear regression, Lasso, Ridge, ElasticNet, Logistic regression, etc. Commented Aug 14, 2020 at 15:44
• For Lasso, see stats.stackexchange.com/q/69568/232706 Commented May 18, 2022 at 18:43
• Commented Sep 7, 2022 at 13:50

With penalized linear models though, there will be a difference. Since the standard deviation of a binary variable is at most $$1/2$$, you'll be increasing the overall scale of the variable by standardizing. That will cause the unpenalized coefficient to decrease in magnitude, which will change the balance of how the penalty applies to different features. I suspect there is no "better" approach then: sometimes the penalty improves performance when the dummies are scaled, and sometimes degrades performance.