What is the difference between least-squares SVM (LS-SVM) and proximal SVM (P-SVM)? How does the decision boundary change in case of both of these types of SVM?
Find this research paper for more info: A Large Dimensional Analysis of Least Squares Support Vector Machines
Least squares support vector machine (LS-SVM) is a successful method for classification or regression problems, in which the margin and sum square errors (SSEs) on training samples are simultaneously minimized. However, LS-SVM only considers the SSEs of input variable.
Linear separation of the datapoints into two classes
Proximal support vector machines and related approaches (Fung & Mangasarian, 2001; Suykens & Vandewalle, 1999) can be interpreted as ridge regression applied to classification problems (Evgeniou, Pontil, & Poggio, 2000). Extensive computational results have shown the effectiveness of PSVM for two-class classification problems where the separating plane is constructed in time that can be as little as two orders of magnitude shorter than that of conventional support vector machines.
When PSVM is applied to problems with more than two classes, the well known one-from-the-rest approach is a natural choice in order to take advantage of its fast performance. However, there is a drawback associated with this one-from-the-rest approach. The resulting two-class problems are often very unbalanced, leading in some cases to poor performance.More
Compared to two widely-used variants of support vector machine for regression, namely, least-square support vector machine (LS-SVM) and proximal support vector machine (P-SVM), ELM is subject to fewer and milder optimization constraints.More info
Least squares support vector machines were first introduced by Suykens and Vandewalle in 1999. They are slightly prior to proximal SVM proposed by Fung and Mangasarian in 2001. It appears that the latter did not know about LS-SVM and re-discovered the approach in a very similar manner.
Both methods are very close in the aspect that, instead of solving a quadratic programming (quadratic problem optimization with linear constraints), they solve a linear system. The trick to do this is to transform the optimization constraint from an inequation to an equation. Both methods use the same trick.
There appear to be minor differences though. In the conclusion of their paper on P-SVM, Fung and Mangasarian note:
Least squares are also used in  to construct an SVM, but with the explicit requirement of Mercer’s positive definiteness condition , which is not needed here. Furthermore, the objective function of the quadratic program of  is not strongly convex like ours. This important feature of PSVM influences its speed as evidenced by the many numerical comparisons given here but not in .
Where  is the original LS-SVM paper by Suykens and Vandewalle.
As of today, a quick internet search shows that LS-SVM are much more common. Whether it is because of anteriority / initial popularity, because they perform better, and/or because their implementation is easier, however, I don't know.