I am using raw data set with 4 feature variables to do a Binominal Classification using Logistic Regression Algorithm.
I made sure that the class counts are balanced. i.e., an equal number of occurrences per class.
The four features are Total Cholesterol, Systolic Blood Pressure, Diastolic Blood Pressure, and Cigraeette count. The class variable name is Stroke.
Dataset description is shown below:
TOTCHOL SYSBP DIABP CIGPDAY STROKE
count 200.000 200.000 200.000 200.000 200.000
mean 231.040 144.560 81.400 4.480 1.500
std 42.465 23.754 11.931 9.359 0.501
min 112.000 100.000 51.500 0.000 1.000
25% 204.750 126.750 73.750 0.000 1.000
50% 225.500 141.000 80.000 0.000 1.500
75% 256.250 161.000 90.000 4.000 2.000
max 378.000 225.000 113.000 60.000 2.000
SKEW is
TOTCHOL 0.369
SYSBP 0.610
DIABP 0.273
CIGPDAY 2.618
STROKE 0.000
Python + sklearn is used here. The problem is that the classification performance gets very negatively-impacted when I try to normalize the dataset using
X=preprocessing.StandardScaler().fit(X).transform(X)
or
X=preprocessing.MinMaxScaler().fit_transform(X)
The classification report (before) normalizing the dataset:
precision recall f1-score support
1 0.85 0.79 0.81 28
2 0.82 0.88 0.85 32
avg / total 0.83 0.83 0.83 60
While the classification report (After) normalizing the dataset:
precision recall f1-score support
1 0.47 1.00 0.64 28
2 1.00 0.03 0.06 32
avg/total 0.75 0.48 0.33 60
Please note that the class variable includes the outcomes 1(yes) and 2(no) instead of the usual 0 and 1.
at this point, I see the preprocessing have damaged the classification accuracy instead of helping it. Is there any logical explanation for that? do I "have" to do the normalization step?
Another remark is the probability scores that I get (before) normalizing the dataset are much higher per class as shown below:
[ 0.65929838 0.34070162]
[ 0.40999878 0.59000122]
[ 0.43592976 0.56407024]
[ 0.40306785 0.59693215]
[ 0.92748002 0.07251998]
[ 0.74173761 0.25826239]
compared to the ones (after) normalizing the dataset
[ 0.51636816 0.48363184]
[ 0.5183946 0.4816054 ]
[ 0.51410135 0.48589865]
[ 0.50739794 0.49260206]
[ 0.52645649 0.47354351]
[ 0.5308564 0.4691436 ]
is there any explanation why I get such results?