0
$\begingroup$

I am confused as to the pros and cons of two different approaches to normalization: Min-Max Scaling, and what the lecturer in the course I am taking refers to as 'Simple Feature Scaling'. The latter he defined to be Xnew = Xold/Xmax, and the former is defined to be: Xnew = (Xold-Xmin)/(Xmax-Xmin). I am looking for a simple code example that demonstrates that the former is more sensitive to outliers.

Before I bother the kind folks on stackexchange I usually ask chatGPT and often get useful answers. On this occasion, chatGPT told me that Min-Max Scaling was more sensitive to outliers than 'Simple Feature Scaling'.

That seemed reasonable, so I asked for an illustrative code example, with a plot. chatGPT gave me various versions of the code below, but in each case there was no visual difference between the plots that I could detect. I am hoping human expertise could help me in tweaking the code snippet below with some data that would illustrate the trade-off that chatGPT said exists between the two approaches. (Assuming chatGPT was right on that score). Thanks in advance.. Code is show below:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

# Create a DataFrame with a feature and introduce outliers
data = {'Feature': [1, 2, 3, 4, 5, 6, 100, 120]}
df = pd.DataFrame(data)

# Min-Max Scaling
min_max_scaled = (df - df.min()) / (df.max() - df.min())

# Max Scaling
max_scaled = df / df.max()

# Plot the original, Min-Max Scaled, and Max Scaled data
plt.figure(figsize=(10, 6))

plt.subplot(3, 1, 1)
plt.scatter(df.index, df['Feature'], color='blue')
plt.title('Original Data')
plt.xlabel('Index')
plt.ylabel('Feature')

plt.subplot(3, 1, 2)
plt.scatter(min_max_scaled.index, min_max_scaled['Feature'], color='green')
plt.title('Min-Max Scaled Data')
plt.xlabel('Index')
plt.ylabel('Feature')

plt.subplot(3, 1, 3)
plt.scatter(max_scaled.index, max_scaled['Feature'], color='red')
plt.title('Max Scaled Data')
plt.xlabel('Index')
plt.ylabel('Feature')

plt.tight_layout()
plt.show()

Here is what the plot looks like when I run the code:

enter image description here

$\endgroup$

1 Answer 1

1
$\begingroup$

The two methods I'm most familiar with are min-max scaling, and standard scaling. Both are designed to remap the range min-max to something "better". Min-max scaling subtracts the min and divides by the max-min just as you have above. Standard scaling subtracts the mean and divides by the std deviation so that you end up with a std deviation of 1 in the scaled data. Now imagine that almost all of your data is between 0 and 1, and you have a single outlier of 10,000,000 (or any really big number). In both min-max scaling, and in your max scaling above, all of your data will be divided by 10,000,000, effectively crushing everything down to close to 0, all because of a single outlier data point. If we have

data = {'Feature': [1, 2, 3, 4, 5, 6, 7, 10000000]} 

then both min-max and max scaling will crush most of the data points close to 0 except for the max. enter image description here

In standard scaling, this won't happen as you divide by the standard deviation.

#std scaling
std_scaled = (df - df.mean()) / df.std()

To contrast max scaling with min-max scaling, consider what happens when your outlier is the min. Max scaling will crush all of your other data, because of the min outlier, whereas min-max scaling will keep things between 0 and 1. Changing data to data = {'Feature': [-10000, 2, 3, 4, 5, 6, 7, 8]} enter image description here

$\endgroup$
4
  • $\begingroup$ thanks ! your answer clearly shows where min-max scaling leaves us in a nice range from 0 to 1 when we throw in the extreme negative value. That was the magic touch to show how the simpler 'divide-by-max' approach does not give us the nice zero to 1 range. Hats off ;^) ! $\endgroup$ Mar 3 at 5:10
  • $\begingroup$ Re-reading, I noticed you said : "In both min-max scaling, and in your max scaling above, all of your data will be divided by 10,000,000, effectively crushing everything down to close to 0". Maybe we want to edit this? because for me the light bulb went on when i noticed that in the simpler 'max scaling' (divide-by-max) the lowest value ends up in a range well below zero (namely, -1250.0,). So the lowest value ended up at the low end of the new range which was well well below zero. Would you agree ? Just want to be sure I have this straight in my head.. thnx again ! $\endgroup$ Mar 3 at 5:14
  • $\begingroup$ If you're dealing with a single outlier and it is the max, then min-max scaling and max-scaling will have very similar behavior, effectively squashing the other data points. If your outlier is the min, this is where the difference between min-max scaling and max scaling is most apparent. $\endgroup$ Mar 3 at 10:21
  • $\begingroup$ Crystal clear now ;-) thnx ! $\endgroup$ Mar 4 at 16:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.