Most articles and textbooks say that soft margin SVM is used is the data is messy/not linearly separable. We introduce slack variables to make the data linearly separable. Kernels are used when the data is nonlinearly separable. In other words, to me it is the same thing: using kernels when the data is not linearly separable. Based on my understanding kernels are used to map datasets into higher dimensions so that they could be linearly separable.

Confusion: I don't understand the difference between soft svm and kernel svm when both these methods are used for non-linearly separable data. Why do we need kernel trick when soft svm is able to handle non-linearly separable data by introducing slack variables? Is it because these are different approaches? Then what are their advantages if they are different approaches? what is the advantage of soft svm compared to using kernels?


1 Answer 1


We introduce slack variables to make the data linearly separable

The above statement is a bit confusing.

A soft margin allows for classes that have a bit of overlap, and can't be completely separated even with higher dimensions. You can think of it as being robust to noise or outliers.

If the groups cannot be separated even by softening the margin then the kernel trick can help find what truly distinguishes your groups, as the current feature space is not able to separate them.

This image has a simple example of where softening margins won't help. You need to add dimensions: Image


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.