Support Vector Machine Errors

Question

What is the difference between SVM classification error, SVM margin error, and SVM total error ? Is there any clear definition for them ? And what is C parameter in SVM ? Its totally confusing me !!!

Answer 1

A SVM has 3 very important components: the support vectors, the separating hyperplane and the margin.

• When a missclassification occurs, it is because a given point is on the wrong side of the separating hyperplane, and that's called a classification error.
• Whenever a point is inside the margin, that counts as a margin error.
• The total error of a SVM, is the sum of the classification error and the margin error.

Now, to understand the C parameter, you've got to know that there are 2 types of SVMs:

• Hard margin: these try to maximize the margin without introducing any kind of errors.
• Soft margin: which also share the same objective, but they allow for some classification and margin errors to occur. The number of errors allowed is controlled by the C parameter (which is often called the penalty parameter): if C is small, the SVM allows for some errors and therefore reaches better generalization; but as C increases, the SVM penalizes these errors more and more, eventually reaching the point that it allows no errors at all.

As you can see bellow on the left, a very large C restricts the model by not allowing any errors, showing very bad results when outliers are present. On the right, by using a small penalty parameter, we let that negative outlier to be in the positive area, therefore maximizing the margin value and reaching a more respectful separating hyperplane.

Although, observe that if you have a soft margin SVM with a very large C value, it will behave just like a hard margin SVM.