Why does an SVM model store the support vectors, and not just the separating hyperplane?

Question

In every explanation of SVMs, we're shown how training finds a hyperplane that best separates the data. Presumably then for inference, you just check which side of the plane a point is on.

However, all the "disadvantages of SVMs" posts [1, 2] lament that SVM models are large and slow because they end up storing most of the data as support vectors.

Why would SVMs store any of the data, rather than just the (coefficients of the) separating hyperplane? (And what is a "support vector" in the soft-margin case, when points of both classes are scattered on both sides of the hyperplane, anyway?)

Answer 1

The hyperplane is a linear combination of the support vectors. In the soft margin case, there is only a limited amount of slack; every input does not get to be support vector. In the nonlinear case, the separating hypersurface may be embedded in an infinite-dimensional space, making it impossible to store. To borrow from the Wikipedia article, the normal vector $w$ is given by

$$w = \sum_i c_i y_i \phi(x_i)$$

where $\phi$ is the feature embedding function, and $c_i$ is a Lagrangian dual variable that is zero for points on the correct side of the margin. Instead, test points are classified through a kernel function $k(x_i,x_j) = \left< \phi(x_i), \phi(x_j) \right>$ like so:

$$x \to \mathrm{sgn}(\left<w , \phi(x)\right> + b) \equiv \mathrm{sgn} \left( b+\sum_i c_i y_i k(x_i, x)\right)$$

Notice how we avoided explicitly calculating $w$.

Answer 2

I would be very grateful if I could receive some help regarding generating hyperplane equation. I need to generate an equation for hyperplane, I have two independent variables and one binary dependent variable.

Regarding this following equation for svm , f(x)=sgn( sum_i alpha_i K(sv_i,x) + b )

I have two independent variables (say P and Q) with 130 point values for each variable. I used svm radial basis function for binary classification (0 and 1) and I calculated for radial basis kernelized case,and now I have one column of 51 y (i) alpha (i) or (dual coeffficients), two columns of 51 sv (support vectors)for P and Q, and one single value for b . I received these using scikit SVC.

https://scikit-learn.org/stable/modules/svm.html

So, how can I generate the equation now? Can I multiply those 51 y (i) alpha (i) or (dual coeffficients) with 51 sv (support vectors) for each variable P and Q so that I have two coefficients for P and Q so that finally my equation appears as : f(x)=sgn( mP + nQ +b) where m = sum of the (product of 51 sv of P with 51 dual coefficients) and n = sum of the (product of 51 sv of Q with 51 dual coefficients). i would be grateful for any kind of suggestion. Many thanks in advance.