2
$\begingroup$

I want to understand the kernel selection rationale in SVM.

Some basic things that I understand is if data is linear, then we must go for linear kernel and if it is non-linear, then others.

But the question is how to understand that the given data is linear or not, especially when it has many features.

I know that by cross validation I can try and feed different kernels and see the output whichever performs best to be selected, but I'm looking for something anyway to have some early indications.

$\endgroup$
2
$\begingroup$

Start with linear kernel and see if your data is linearly seperable or not. Performing that is simpler than looking for early indications.

Linear kernels are suggested when the number of features is larger than the number of observations in the dataset (otherwise RBF will be a better choice).

However, once you conclude that you have non-linear data, you can try to visualize and map the non-linear separable data into a higher dimensional space and see if it makes the data linearly separable. That is what a kernel does anyway and so you visualizing will give you insights and indications about the type of kernel to be used.

enter image description here

This link might help.

$\endgroup$
2
  • $\begingroup$ Thanks for reply @Sharan, but how to identify if the data is linearly separable or not? Also visualization would help only if we have 2 or 3 features in data set otherwise with high number of features how you will plot it? However your suggestion "Linear kernels are suggested when the number of features is larger than the number of observations in the dataset (otherwise RBF will be a better choice)." is helpful. $\endgroup$
    – SKB
    Jan 10 '20 at 7:42
  • $\begingroup$ Empirically test it by trying to achieve high accuracy on training data using a linear classifier such as linear regression. If you have considerably lower accuracy then it indicates that that your data is linearly not seperate. Or start with a linear kernel and see. $\endgroup$
    – Sharan
    Jan 10 '20 at 9:00
0
$\begingroup$

This paper by Chih-Wei Hsu et al. is a good starting point for kernel selection. On page 3, they suggest using RBF kernel and fine tuning it.

Andrew Ng has also provided a very high level thumb rule in his video on SVM: If number of observations are larger than features, use Gaussian kernel. Use linear otherwise.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.