1
$\begingroup$

Currently I'm working in WEKA, using the SMO classifier (an implementation of SVM). For an assignment I am requested to use a polynomial kernel, and report the results for degrees varying from 1 up to 50. I need to comment on overfitting, and my understanding is that higher degrees lead up to a higher risk of overfitting. (And hence, the percentage of correctly classified instances, using 10-fold cross validation should increase as the degree increases).

My intuition is right for the 'first' degrees, resulting in a 99.6% rate at degree=10. However, when I set the degree to 50, WEKA reports a 9% rate.

How is it possible that this rate drops (and by so much)?

$\endgroup$
1
$\begingroup$

Setting the polynomial kernel degree to 50 is likely causing the SVM to severely overfit to the data, which would explain the 9% you are seeing. Increasing the degree helps the SVM make an appropriate generalization, but when you start to see the validation/test accuracy decrease, then the SVM is starting to overfit.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.