13
$\begingroup$

I understand what Standard Scalar does and what Normalizer does, per the scikit documentation: Normalizer, Standard Scaler.

I know when Standard Scaler is applied. But in which scenario is Normalizer applied? Are there scenarios where one is preferred over the other?

| improve this question | |
$\endgroup$
  • $\begingroup$ You don't always need to use either: It's also worth adding that tree-based classifier/regressor algorithms (RF/XGB/GBT) don't need standardization, you can just feed them the raw data. (You might still choose to do standardization anyway, e.g. for plotting, correlation, measures of association) $\endgroup$ – smci Jul 9 '19 at 11:13
9
$\begingroup$

They are used for two different purposes.

StandardScaler changes each feature column $f_{:,i}$ to $$f'_{:,i} = \frac{f_{:,i} - mean(f_{:,i})}{std(f_{:,i})}.$$

Normalizer changes each sample $x_n=(f_{n,1},...,f_{n,d})$ to $$x'_n = \frac{x_n}{size(x_n)},$$ where $size(x_n)$ for

  1. l1 norm is $\left \| x_n \right \|_1=|f_{n,1}|+...+|f_{n,d}|$,
  2. l2 norm is $\left \| x_n \right \|_2=\sqrt{f^{2}_{n,1}+...+f^{2}_{n,d}}$,
  3. max norm is $\left \| x_n \right \|_\infty=max\{|f_{n,1}|,...,|f_{n,d}|\}$.

To illustrate the contrast, consider data set $\{1, 2, 3, 4, 5\}$ which is one dimensional (each data point has one feature),
After applying StandardScaler, data set becomes $\{-1.41, -0.71, 0. ,0.71, 1.41\}$.
After applying any type of Normalizer, data set becomes $\{1., 1., 1., 1., 1.\}$, since the only feature is divided by itself. So Normalizer has no use for this case. It also has no use when features have different units, e.g. $(height, age, income)$.

As mentioned in this answer, Normalizer is mostly useful for controlling the size of a vector in an iterative process, e.g. a parameter vector during training, to avoid numerical instabilities due to large values.

| improve this answer | |
$\endgroup$
2
$\begingroup$

  • StandardScaler : It transforms the data in such a manner that it has mean as 0 and standard deviation as 1. In short, it standardizes the data. Standardization is useful for data which has negative values. It arranges the data in a standard normal distribution. It is more useful in classification than regression. You can read this blog of mine.

  • Normalizer : It squeezes the data between 0 and 1. It performs normalization. Due to the decreased range and magnitude, the gradients in the training process do not explode and you do not get higher values of loss. Is more useful in regression than classification. You can read this blog of mine.

| improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ The normalizer you have defined in your blog is MinMax scaler. The link which I have put for normalization is different. It makes the l2 norm of each data row equal to 1. $\endgroup$ – Heisenbug Feb 21 '19 at 5:53
  • $\begingroup$ This answer may help you. $\endgroup$ – Shubham Panchal Feb 21 '19 at 6:42
  • 8
    $\begingroup$ -1: "[standardization] arranges the data in normal distribution." you should clarify what you mean by this. I read this as "standardization transforms data to have the normal distribution", which is not true. You should also explain why standardization is more useful in classification than regression (and vice versa for normalization); I doubt that claim. $\endgroup$ – Artem Mavrin Feb 25 '19 at 18:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.