What is "data scaling" regarding StandardScaler()?

Question

I'm trying to figure out the purpose of StandardScaler() in sklearn.

The tutorial I am following says

"Remember that you also need to perform the scaling again because you had a lot of differences in some of the values for your red and white [wines]"

So I looked up the function in the sklearn docs.

"Standardize features by removing the mean and scaling to unit variance"

https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

• What good would removing the mean do?
• What is scaling the data? Hard to google that.

# Scale the data with `StandardScaler`
X = StandardScaler().fit_transform(X)

Answer 1

I will use k-Nearest Neighbor algorithm to explain why we must do scaling as a preprocessing step in most machine learning algorithms.

Let's say you are trying to predict if I transaction is fraudulent or not, that is, you have a classification problem and you only have two features: value of transaction and time of the day. Both variables have different magnitudes, while transactions can vary from 0 to 100000000 (It is just an example), and time of the day between 0 to 24 (let's use only hours).

So, while we are computing the nearest neighbor, using euclidean distance, we will do

distance_class = sqrt(
    (new_value_transaction - old_value_transaction)**2) + 
    (new_time_of_day - old_time_of_day)**2)
)

Where old is the reference to our train data and new is related to a new transactions we want to predict the class.

So now you can see that transactions will have a huge impact, for example,

new_value_transaction = $100
new_time_of_day = 10

old_value_transaction = $150
new_time_of_day = 11

class_distance = sqrt(($50)**2) + (1)**2)

Now, you have no indication that transaction value is more important than time of the day, that is why we will scale our data.

Between the alternatives, we can have a lot of different, such as MinMaxScaler, StandardScaler, RobustScaler, etc. Each of them will treat the problem different. To be honest? Always try to use at least two of them to compare results.

You can read more about in the sklearn documentation: https://scikit-learn.org/stable/modules/preprocessing.html#preprocessing

I hope you got the feeling why we should use standardization techniques. Let me know if you have any further questions.

To complement, here is a visual explanation of what I explained above.

Credit: https://www.youtube.com/watch?v=d80UD99d4-M&list=PLpQWTe-45nxL3bhyAJMEs90KF_gZmuqtm&index=10

In the video they give a better explanation. Also, I highly recommend this course, the guys are amazing.

Answer 2

If your data has more than one explanatory variables it may be important to how they distributed. They may distribute differently or they may have a same distribution. When they ditribute differently it will be a problem for estimation. Especially, regression-based methods assume that data comes from a "Normal Distribution". These algorithms need a variance but not too much variance, because it will be really hard to predict outcomes consistently. So, you need to scale your data before building your model.

Not only for distribution but also for computation it is a good idea to scale your data.

Also, you can have look at this article:

https://medium.com/greyatom/why-how-and-when-to-scale-your-features-4b30ab09db5e