Currently I'm working in WEKA, using the SMO classifier (an implementation of SVM). For an assignment I am requested to use a polynomial kernel, and report the results for degrees varying from 1 up to 50. I need to comment on overfitting, and my understanding is that higher degrees lead up to a higher risk of overfitting. (And hence, the percentage of correctly classified instances, using 10-fold cross validation should increase as the degree increases).
My intuition is right for the 'first' degrees, resulting in a 99.6% rate at degree=10. However, when I set the degree to 50, WEKA reports a 9% rate.
How is it possible that this rate drops (and by so much)?