0
$\begingroup$

Concering Support Vector Machines (SVM): it is always mentioned that $\textbf{w}^T\textbf{x}_i - b >= 1$, for $\textbf{x}_i$ of class 1 (i.e. $y=1$) and $\textbf{w}^T\textbf{x}_i - b <= -1$, for $\textbf{x}_i$ of class -1 (i.e. $y=-1$) from the Wikipedia page I read that it is related to normalization of the dataset, so I guess all the $\textbf{x}_i$ vectors are normalized (divided by) their norm ie $\textbf{x}_i$ := $\textbf{x}_i/|| \textbf{x}_i||_2$ so they all have norm 1 and projecting i.e. doing the scalar product $\textbf{w}^T\textbf{x}_i$ would give someting lower than 1 or 1 if $\textbf{w}$ was also normalized (which maybe it is not in general and so all this is not so clear?)

So I would like a detailed (don't hesitate to put all "so-called" "trivial" steps!) explanation of why this is so? Also if possible a geometric explanation/drawing.

Second question is about $b$. For a line in 2D it would represent the y-intercept (only if the w coefficient of the parameter corresponding to y in the vector w would be 1, otherwise need to divide b by it...) ... so i m not completely sure what distance it would be geometrically... can anyone draw this clearly? It's a shape that these specific points I mention seem to be always missing (or treated as "trivial" which they are not!) in any lectures I've seen...

$\endgroup$
1
$\begingroup$

It's -1 and +1 because the support vectors are normalized to ensure that the cases closest to the hyperplane have a distance of exactly 1. It's exactly one support vector distance from the plane.

The intercept of the plane is just the value of the plane if all features are zero (or absent). It does not have a practical meaning (except that it is negative if the mode is 0 and positive if it is 1) and is set to the point that ensures that the plane has exactly one SV distance from the closest cases and intersects the cases at the optimal point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.