SVM model classifying into one class only, after standardization

Question

I'm trying to use SVM in R (e1071 package) to classify samples as normal or tumor. I have two separate data sets - Training (~50 samples, 100 features) and Test (~60 samples). These data sets are microarray expression values from two different microarray platforms performed by two different research groups and so their value ranges are very different. I am new to SVM and R, and so at first, I did a Z-score standardization on my Training set and then I did a Z-score standardization on my Test set separately. i.e.

(x_Train - mean_Train) / (stdev_Train)

(x_Test - mean_Test) / (stdev_Test)

As I explored more forum posts on this topic, I see that it is suggested to use the parameters from Training set on my Test set

(x_Test - mean_Train) / (stdev_Train)

So that my SVM model is more generalizable and can be applied to new samples.

Here is the problem,

When I standardized the sets separately, my best model (after 10-fold cross validation using the tune function) had around 75% accuracy classifying the test set. However, after standardizing as suggested with my Training parameters the accuracy of my model to classify the Test set dropped drastically to 25% using the same features.

I tried using different combinations of features to see if it makes a difference and I saw that no matter what I had changed, all samples from the Test set were classified into one class (all in normal, and zero in tumor), while my previous model with separate standardization classified samples into both classes.

Here is an example of one of the features

Training set (raw)
         Feature 1     
Sample 1 25.82977   
Sample 2 42.62437   
Sample 3 28.91158   
Sample 4 53.5708    
Sample 5 30.92296
Sample 6 99.16994   
Sample 7 40.75973

Test set (raw)
         Feature 1
Sample 1 2.885865028    
Sample 2 2.572860413    
Sample 3 2.809136715    
Sample 4 2.259630716    
Sample 5 2.797155715    
Sample 6 2.439700763    
Sample 7 2.197087754

Training set (standardized)
         Feature 1
Sample 1 -0.795137358
Sample 2 -0.132081654
Sample 3 -0.673466601
Sample 4  0.300086594
Sample 5 -0.594056732
Sample 6  2.100353934
Sample 7 -0.205698184

Test set (standardized) 
         Feature 1 
Sample 1 -1.700969415
Sample 2 -1.713326928
Sample 3 -1.703998671
Sample 4 -1.725693328
Sample 5 -1.704471684
Sample 6 -1.71858411
Sample 7 -1.728162542

I know SVM uses a threshold to classify samples into either normal or tumor. From what I showed above, their ranges are completely different, is that what is affecting its decision?

What is the problem here and how can I tackle it, thank you in advance. I am so lost and need every bit of advice!

*Edit: As you can see the values of Training set have a completely different range from the values of the Test set, can I still use the mean and standard deviation from my Training set to scale the corresponding features in the Test set?

Answer 1

I can't add a comment so I'll post this here.

My main question is why are your feature values so wildly different different between you your test and training sets?

For example the raw values of feature 1.

Training set: All values > 20, looks like they average about 35-40.
Test set: All values around 2, looks like they all fall between 2-3.

What is the reason for this. Also have you shuffled you data before performing a train test split? The you should plot a distribution of the feature values for both your training and test sets sets, and make sure they are somewhat similar.

It looks like you you have a bad covariate shift between training and test sets. Don't worry about this yet as I think it's a data preprocessing problem. First answer the question if the distribution of values of each feature are similar between training and test sets.