Why is a MinMax Scaler scaling each column independently? Isn't it losing information if the values are somehow connected? If the value in column B is always an upper limit for the value in column C, after scaling the value in column B will be smaller than the value in column C because the range is much smaller.
I can't get my head around why that makes any sense. Shouldn't it scale all columns on the same scale to keep the relation between them?
When we say two values are 'connected', we are typically talking about correlation (or covariance).
The correlation between variables A and B is conserved across linear scalings (which MinMax Scalers perform).
For example, if A = [1,5,10], and B = [10,50,100], their Pearson correlation coefficient (i.e. how 'connected' they are) would be 1.
If we rescaled both using a MinMax scaler with bounds [0,1], they would now have the same values (i.e. A=[0.1, 0.5, 1.0] and B=[0.1, 0.5, 1.0]), but more importantly, their Pearson correlation coefficient would still be equal to 1.
This is why a MinMax Scaler can be applied to each column/feature independently.
Again, this is only the case for linear scalings. Non-linear scalers are not guaranteed to conserve the covariance structure of your data.
Most models and theory suppose features are independant variables. Moreover some models may take into acccount only biger values, so scaling is important and necssary. Scaling each one independently assures each feature has same range of value, and so the same importance while training your model.
