2
$\begingroup$

I want to understand the kernel selection rationale in SVM.

Some basic things that i understand is if data is linear then we must go for linear kernel and if it is non-linear then others.

But question is how to understand that the given data is linear or not(specially when it have many features).

I know by cross validation i can try and feed different kernels and see the output, whoever performs best to be selected.

But anyway to have some early indications ?

$\endgroup$
2
$\begingroup$

Start with linear kernel and see if your data is linearly seperable or not. Performing that is simpler than looking for early indications.

Linear kernels are suggested when the number of features is larger than the number of observations in the dataset (otherwise RBF will be a better choice).

However, once you conclude that you have non-linear data, you can try to visualize and map the non-linear separable data into a higher dimensional space and see if it makes the data linearly separable. That is what a kernel does anyway and so you visualizing will give you insights and indications about the type of kernel to be used.

enter image description here

This link might help.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ Thanks for reply @Sharan, but how to identify if the data is linearly separable or not? Also visualization would help only if we have 2 or 3 features in data set otherwise with high number of features how you will plot it? However your suggestion "Linear kernels are suggested when the number of features is larger than the number of observations in the dataset (otherwise RBF will be a better choice)." is helpful. $\endgroup$ – Saurabh Jan 10 at 7:42
  • $\begingroup$ Empirically test it by trying to achieve high accuracy on training data using a linear classifier such as linear regression. If you have considerably lower accuracy then it indicates that that your data is linearly not seperate. Or start with a linear kernel and see. $\endgroup$ – Sharan Jan 10 at 9:00
0
$\begingroup$

This Paper by Chih-Wei Hsu et All is a good starting point for kernel selection. On page 3 they suggest using RBF kernel and fine tune it.

Andrew Ng also provide a very high level thumb rule in his video on SVM - If number of observations are larger than features use Gaussian kernel. Use linear otherwise.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.