What is the difference between LS-SVM and P-SVM?

Question

What is the difference between least-squares SVM (LS-SVM) and proximal SVM (P-SVM)? How does the decision boundary change in case of both of these types of SVM?

Answer 1

Find this research paper for more info: A Large Dimensional Analysis of Least Squares Support Vector Machines

Least squares support vector machine (LS-SVM) is a successful method for classification or regression problems, in which the margin and sum square errors (SSEs) on training samples are simultaneously minimized. However, LS-SVM only considers the SSEs of input variable.

Least-squares SVM

Linear separation of the datapoints into two classes

Implementation and Equations

Proximal support vector machines and related approaches (Fung & Mangasarian, 2001; Suykens & Vandewalle, 1999) can be interpreted as ridge regression applied to classification problems (Evgeniou, Pontil, & Poggio, 2000). Extensive computational results have shown the effectiveness of PSVM for two-class classification problems where the separating plane is constructed in time that can be as little as two orders of magnitude shorter than that of conventional support vector machines.

When PSVM is applied to problems with more than two classes, the well known one-from-the-rest approach is a natural choice in order to take advantage of its fast performance. However, there is a drawback associated with this one-from-the-rest approach. The resulting two-class problems are often very unbalanced, leading in some cases to poor performance.More

Compared to two widely-used variants of support vector machine for regression, namely, least-square support vector machine (LS-SVM) and proximal support vector machine (P-SVM), ELM is subject to fewer and milder optimization constraints.More info

Fuzzy Least Squares Twin Support Vector Machines

Proximal support vector machine classifiers

Answer 2

Least squares support vector machines were first introduced by Suykens and Vandewalle in 1999. They are slightly prior to proximal SVM proposed by Fung and Mangasarian in 2001. It appears that the latter did not know about LS-SVM and re-discovered the approach in a very similar manner.

Both methods are very close in the aspect that, instead of solving a quadratic programming (quadratic problem optimization with linear constraints), they solve a linear system. The trick to do this is to transform the optimization constraint from an inequation to an equation. Both methods use the same trick.

There appear to be minor differences though. In the conclusion of their paper on P-SVM, Fung and Mangasarian note:

Least squares are also used in [33] to construct an SVM, but with the explicit requirement of Mercer’s positive definiteness condition [35], which is not needed here. Furthermore, the objective function of the quadratic program of [33] is not strongly convex like ours. This important feature of PSVM influences its speed as evidenced by the many numerical comparisons given here but not in [33].

Where [33] is the original LS-SVM paper by Suykens and Vandewalle.

As of today, a quick internet search shows that LS-SVM are much more common. Whether it is because of anteriority / initial popularity, because they perform better, and/or because their implementation is easier, however, I don't know.