# How to normalize the data correctly in spam dataset

• I'm working on the spam dataset to classify the inputs into binary classes.
• my problem is that: the observations in the dataset are floats small numbers in the first 53 column, and the 54 is float larger numbers, while the last two columns are integers.

### My Question:

How to Normalize this dataset correctly, so all the observations have the same importance?

import pandas as pd

• one proposed approach which didn't seem very convenient to me, because it normalize the whole row input is that:
#========================
# Normalization Function
#========================
def Normalize(x):
'''
==================================
Normalization Function
==================================
-----------
Parameters:
-----------
@Parameter x: Vector
---------
Returns:
---------
Normalized Vector.
================================
'''
norm=0.0
for e in x:
norm+=e**2
for i in range(len(x)):
x[i]/=sqrt(norm)
return x


Normalizing so that "all the observations have the same importance" is kinda ambiguous and ill-defined. In any case, it would be strongly advised to avoid re-inventing the wheel, and use one of the several scalers available out there (e.g. in the sklearn.preprocessing module).

Here is an example using MinMaxScaler, which will re-scale your data in [0, 1] column-wise:

import pandas as pd
# result:
0     1     2    3     4     5   ...     52     53     54   55    56  57
0  0.00  0.64  0.64  0.0  0.32  0.00  ...  0.000  0.000  3.756   61   278   1
1  0.21  0.28  0.50  0.0  0.14  0.28  ...  0.180  0.048  5.114  101  1028   1
2  0.06  0.00  0.71  0.0  1.23  0.19  ...  0.184  0.010  9.821  485  2259   1
3  0.00  0.00  0.00  0.0  0.63  0.00  ...  0.000  0.000  3.537   40   191   1
4  0.00  0.00  0.00  0.0  0.63  0.00  ...  0.000  0.000  3.537   40   191   1

[5 rows x 58 columns]

from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler() # define the scaler
df_scaled = pd.DataFrame(sc.fit_transform(df)) # fit & transform the data