# Support vector machine margin term, norm or norm squared?

$$\lambda||\hat w||^2 +(1/n)\sum max(0,1-y_i(\hat w \hat x_i -b))$$

we know that $2/||\hat w||$ is the width of the margin.

The second term penalizes a misclassified point for how far away it is from the margin relative to the width of the margin. E.g., suppose there is a misclassified point $x_0$: $$1-y_0(\hat w \hat x_0 -b)=3$$ That means $x_0$ is $3/||\hat w||$ away from $1-y_i(\hat w \hat x_i -b)=0$ and is penalized for $3$.

The first term penalizes for the inverse of the width of the margin squared. I find it hard to reconcile with the second term - they seem to be of different scales. Is there any reason (intuitively) why $||\hat w||^2$ is used instead of just $||\hat w||$?

PS: Perhaps one reason is that $||\hat w||^2$ is easier computation-wise (quadratic programming)? Or perhaps norm squared assumes sample noise to be Gaussian? I am not sure. Has anyone seen the use of $||\hat w||$ instead of $||\hat w||^2$?

It is indeed for computational tractability. You would not lose the all-important sparsity, since that is provided by the second term. The original formulation of SVM, which you can find on Wikipedia actually uses $\lVert w \rVert$:

Minimize $\|{\vec {w}}\|$ subject to ${\displaystyle y_{i}({\vec {w}}\cdot {\vec {x}}_{i}-b)\geq 1}-\xi_i$

Obviously, minimizing $\lVert w \rVert$ is the same as $\lVert w \rVert^2$. A quantative difference arises when introducing the constraint using the Lagrangian function, but qualitatively it is the same.