RBF kernel using SVM depends on two parameters C and gamma. If the equation of the kernel RBF as the following:

$K(X,X')= \exp(\gamma||X-X'||^2)$

In the equation I can see where can I use gamma, but I can't find the C parameter. So, can enybody tell me please?

Thanks in advance,


The C parameter in SVMs doesn't have to do anything with the kernel function. C is the penalty associated to the instances which are either misclassified or violates the maximal margin. As you may know already, SVM returns the maximum margin for the linearly separable datasets (in the kernel space).

It might be the case that the dataset is not linearly separable. In this case the corresponding SVM quadratic program is unsolvable. To make it solvable, we redefine the quadratic program to return the maximum margin with respect to some error cost C for the points violating the maximum margin (misclassified samples also violate the maximum margin). Below you can see a simple example I found on the Internet. This uses linear kernel but the story is the same for other non-linear kernels as well. Non-Linear dataset

The effect C has on the classification result is shown in the picture below:

C effect

As you can see, if you set the error penalty to be very high, you will end up with less misclassifications and fewer support vectors (the highlighted points are SVs). However, if you set it too high when you are using non-linear kernels (specially Gaussian) you might overfit.

If you are interested in the theory behind it, here is what we are trying to optimize:

enter image description here

If we write its dual, we'll get:

enter image description here

See how convenient it is! No C in the kernel function!

In other words, you are mapping the samples to a higher dimensional space using the kernel function, then solving it using some penalty term C.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.