1
$\begingroup$

I am working on a binary class classification problem. Each sample is a vector 1x101, I have a lot of data samples more than 150k I tried training a linear svm and a non-linear svm (RBF) "zscore normalization is used in both cases". surprisingly, the linear does better than the svm (RBF). I am trying to explain this by considering the following points:

  1. I beleive that the quality of my feature is not very good.
  2. I think the nonlinear case experinces a kind of overfitting.

my question is how to explain this behaviour?!! does what I am thinking in make sense?!! I am thinking in using Adaboost to perform the training, is it a good idea or not?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ With that many features and the results that you describe, I would consider it possible that you have not yet found the best parameter regime for your nonlinear SVM. You haven't spoken to the regularization tuning or distance tuning in the RBF. Also, I suggest applying several different amounts of PCA, reducing your input dimensions to 75, 50, 25. Since analogue methods can be significantly effected by noisy examples, its important to tune the model using proper cross validation. I've also seen several cases where PCA performs much better in high variance scenarios than regularization. $\endgroup$
    – AN6U5
    Jun 21, 2016 at 5:08

1 Answer 1

Reset to default
3
$\begingroup$

The quality of your features might actually be better than you think. If they provide linear separability, nonlinear kernels will overfit more readily than a linear kernel, leading to your result.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.