Are there any concerns to normalizing data to be within the range 0 - 1 and mean centering the data as well?

Does it matter which comes first?

If you do one, is the other not required?


If you don't center before you normalize, you don't take advantage of the full [-1,1] range if your input is non-negative. The combination of centering and normalization is called standardization.

Sometimes one normalizes by the standard variation, and other times by just the range (max-min). The latter is called feature scaling. The effect is much the same. Normalizing by the range is easier computationally. Normalizing by the standard deviation fixes the sample variance, which is nice from a statistical perspective. When using the standard deviation, the subtraction is usually against the sample mean rather than the minimum.

There are several reasons for performing standardization. Sometimes we are interested in relative rather than absolute values. Standardization achieves invariance to these irrelevant differences. By explicitly preprocessing the data to reflect this disinterest, we relieve the model from having to learn it, allowing us to use a simpler one. Another reason is computational; it reduces the condition number -- you can think of this as the skewness or niceness of the loss surface -- making optimization easier and faster.

| improve this answer | |
  • $\begingroup$ thanks for the answer. Could I ask a follow up? Would you be able to tell me or point me towards some information for doing the above to a dataset that has two subgroups, and applying the centering and normalization to the entire dataset with respect to only one of the groups? $\endgroup$ – atomsmasher Jul 15 '16 at 18:37
  • $\begingroup$ Read the documentation and code related to sklearn's StandardScaler $\endgroup$ – Emre Jul 15 '16 at 18:39
  • $\begingroup$ from my understanding normalization consisted of $${x_{i} - min(x)} \over {max(x) - min(x)}$$ from that documentation, the standard scalar calculates the standard deviation and the mean and then applys it. isn't that different? $\endgroup$ – atomsmasher Jul 15 '16 at 18:49
  • $\begingroup$ Yes, it is. Thank you for mentioning it; I've edited my answer. $\endgroup$ – Emre Jul 15 '16 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.