Skip to main content

Normalization is a technique often applied as part of data preparation for machine learning. The goal of normalization is to change the values of numeric columns in the dataset to use a common scale, without distorting differences in the ranges of values or losing information.

"Normalization" refers to several related processes:

  • ("Feature scaling") A set of numbers whose maximum is $M$ and minimum is $m$ can be converted to the range from $0$ to $1$ by means of an affine transformation (which amounts to changing their units of measurement) $x \to (x-m)/(M-m)$.

  • A set of positive numbers $\{p_i\}$ representing probabilities or weights can be uniformly rescaled to sum to unity: divide each $p_i$ by the sum of all the $p_i$.

  • Analogously, a distribution (or indeed any non-negative function with a finite nonzero integral) can be normalized to have a unit integral by dividing its values by the integral.

  • A vector in a normed linear space is normalized (to unit length) by dividing it by its norm. This is a general procedure encompassing the two preceding operations as special examples.

The range from $0$ to $1$ can be made from $0$ to any desired limit $\alpha$ by multiplying a previously unit-normalized value by $\alpha$.

Other kinds of operations exist having a similar intent of re-expressing values in a predetermined range. Many of these are nonlinear and tend to be used in specialized settings.