2
$\begingroup$

How do we decide which kernel needs to be used for a particular dataset? Is there any criteria needs to be followed? And also what is the criteria to select C and gamma values?

Kindly excuse me if this question looks silly.

Thank you.

$\endgroup$
  • $\begingroup$ The docs at sklearn will be very helpful... It's the by docs maintained by any ml/DL modules.. $\endgroup$ – Aditya Apr 9 '18 at 18:22
2
$\begingroup$

Without knowing the geometry of your data, determining a kernel is generally achieved through trial and error. The radial basis function (RBF) kernel is a good starting choice because most data are not linearly separable. Fortunately training an SVM is fast, so brute-forcing the kernel search is not a terrible method.

For selecting the C and gamma values, we generally use a grid search:

example from scikit-learn

Every cell in the graph represents a SVM model trained on your data given the respective C and gamma parameters. The color represents its accuracy depicted by the scale on the right. Note that our scales are logarithmic, so we are able to search a broad solution space while only training 169 models.

From this example, we can predict that the optimal (C,gamma) value combination would be in the region around $( 10, 0.1 )$ or $( 10^{10}, 1e^{-7}) $

We can further refine our C and gamma parameters by "zooming" into the regions with greatest validation accuracy. In other words, we can repeat the grid search with finer granularity and with boundaries determined by the regions of interest until we find the optimal C/gamma combination.

$\endgroup$
0
$\begingroup$

Which has a minimum test or CV error (Or maximum accuracy).

$\endgroup$
  • $\begingroup$ Does that mean Trial and Error method? $\endgroup$ – deepguy Apr 9 '18 at 18:43
  • $\begingroup$ In general, yes. Of course, there are some graphical methods, which are useful to determine type of kernel, but this methods are not universal and don't even work in multidimensional problems. $\endgroup$ – D.G Apr 9 '18 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.