Standardization vs min-max scaling

Question

In the book Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow 2nd Edition by Aurélien Géron, the author quoting:

Unlike min-max scaling, standardization does not bind values to a specific range, which may be a problem for some algorithms (e.g., neural networks often expect an input value ranging from 0 to 1)

Does this mean that standardization is not good for neural networks? Please explain how neural networks do not work well with standardization but are good with the min-max scale.

Answer 1

Unlike ML models, DL models require the data to be in it's most raw for to extract the most amount of information from it. Now when you scale all your features in ML models such as SVM it is a necessity because some of the features might have a very large variance relative to other features and therefore they will dominate the other features eventually leading to worse performance. In case of DL you use stochastic methods and adjust the weights and biases that are not affected by the scaling (take this with a grain of salt), the NN actually works best of you do not scale the features. due to the stochastic nature of e.g Gradient descent, we can never be sure what works best. So it's always best to try out with few examples to train and test how well does the model performs and does the scaling helps.

Highly recommend this article I used for my work: https://machinelearningmastery.com/how-to-improve-neural-network-stability-and-modeling-performance-with-data-scaling/