2
$\begingroup$

What is the difference between SVM classification error, SVM margin error, and SVM total error ? Is there any clear definition for them ? And what is C parameter in SVM ? Its totally confusing me !!!

$\endgroup$
  • $\begingroup$ Can you add some context to where you heard or read these terms? $\endgroup$ – Wes Feb 18 '19 at 14:56
4
$\begingroup$

A SVM has 3 very important components: the support vectors, the separating hyperplane and the margin.

components of a SVM

  • When a missclassification occurs, it is because a given point is on the wrong side of the separating hyperplane, and that's called a classification error.
  • Whenever a point is inside the margin, that counts as a margin error.
  • The total error of a SVM, is the sum of the classification error and the margin error.

Now, to understand the C parameter, you've got to know that there are 2 types of SVMs:

  • Hard margin: these try to maximize the margin without introducing any kind of errors.
  • Soft margin: which also share the same objective, but they allow for some classification and margin errors to occur. The number of errors allowed is controlled by the C parameter (which is often called the penalty parameter): if C is small, the SVM allows for some errors and therefore reaches better generalization; but as C increases, the SVM penalizes these errors more and more, eventually reaching the point that it allows no errors at all.

As you can see bellow on the left, a very large C restricts the model by not allowing any errors, showing very bad results when outliers are present. On the right, by using a small penalty parameter, we let that negative outlier to be in the positive area, therefore maximizing the margin value and reaching a more respectful separating hyperplane.

comparison of C values

Although, observe that if you have a soft margin SVM with a very large C value, it will behave just like a hard margin SVM.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.