0
$\begingroup$

I'm trying to figure out the purpose of StandardScaler() in sklearn.

The tutorial I am following says

"Remember that you also need to perform the scaling again because you had a lot of differences in some of the values for your red and white [wines]"

So I looked up the function in the sklearn docs.

"Standardize features by removing the mean and scaling to unit variance"

https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html

  • What good would removing the mean do?
  • What is scaling the data? Hard to google that.
# Scale the data with `StandardScaler`
X = StandardScaler().fit_transform(X)
$\endgroup$
2
$\begingroup$

I will use k-Nearest Neighbor algorithm to explain why we must do scaling as a preprocessing step in most machine learning algorithms.

Let's say you are trying to predict if I transaction is fraudulent or not, that is, you have a classification problem and you only have two features: value of transaction and time of the day. Both variables have different magnitudes, while transactions can vary from 0 to 100000000 (It is just an example), and time of the day between 0 to 24 (let's use only hours).

So, while we are computing the nearest neighbor, using euclidean distance, we will do

distance_class = sqrt(
    (new_value_transaction - old_value_transaction)**2) + 
    (new_time_of_day - old_time_of_day)**2)
)

Where old is the reference to our train data and new is related to a new transactions we want to predict the class.

So now you can see that transactions will have a huge impact, for example,

new_value_transaction = $100
new_time_of_day = 10

old_value_transaction = $150
new_time_of_day = 11

class_distance = sqrt(($50)**2) + (1)**2)

Now, you have no indication that transaction value is more important than time of the day, that is why we will scale our data.

Between the alternatives, we can have a lot of different, such as MinMaxScaler, StandardScaler, RobustScaler, etc. Each of them will treat the problem different. To be honest? Always try to use at least two of them to compare results.

You can read more about in the sklearn documentation: https://scikit-learn.org/stable/modules/preprocessing.html#preprocessing

I hope you got the feeling why we should use standardization techniques. Let me know if you have any further questions.

To complement, here is a visual explanation of what I explained above.

enter image description here

Credit: https://www.youtube.com/watch?v=d80UD99d4-M&list=PLpQWTe-45nxL3bhyAJMEs90KF_gZmuqtm&index=10

In the video they give a better explanation. Also, I highly recommend this course, the guys are amazing.

$\endgroup$
6
  • $\begingroup$ Thank you. So because the difference between (0 and 1000000) is much greater than the difference between (0 and 24)... we need to adjust our 2 columns somehow? What adjustments are made? $\endgroup$ – Kalanos Aug 28 '19 at 14:37
  • $\begingroup$ Exactly. By subtracting the mean and dividing by standard deviation you will see that all your variables will be between a specific range, for example, -2 to 2, and then the machine learning algorithm can treat both with equal weights, in other words, they will have the same importance apriori. $\endgroup$ – Victor Oliveira Aug 28 '19 at 14:51
  • $\begingroup$ Why would you want them to have the same importance when one is clearly more important than the other? Isn't that completely bias'ing the data? $\endgroup$ – Kalanos Aug 28 '19 at 14:54
  • $\begingroup$ Like I said, apriori, then the model will give a new weight to your features and then it will give more importance to one of your features. Bias would happen in the first case where you have a higher magnitude for one feature already. This video will help you to understand that: youtu.be/… Dont hesitate to keep asking. $\endgroup$ – Victor Oliveira Aug 28 '19 at 15:01
  • 1
    $\begingroup$ Thanks so much, I'll check that out. Your diagram helps... you aren't removing any data from the higher magnitude feature... rather you are representing it differently? $\endgroup$ – Kalanos Aug 28 '19 at 15:07
1
$\begingroup$

If your data has more than one explanatory variables it may be important to how they distributed. They may distribute differently or they may have a same distribution. When they ditribute differently it will be a problem for estimation. Especially, regression-based methods assume that data comes from a "Normal Distribution". These algorithms need a variance but not too much variance, because it will be really hard to predict outcomes consistently. So, you need to scale your data before building your model.

Not only for distribution but also for computation it is a good idea to scale your data.

Also, you can have look at this article:

https://medium.com/greyatom/why-how-and-when-to-scale-your-features-4b30ab09db5e

$\endgroup$
1
  • $\begingroup$ So if I have many more red wines than white wines in my wine dataset - then my data is not realistically distributed? $\endgroup$ – Kalanos Aug 28 '19 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.