The optimization goal of the SVM is to maximize the distance between the positive and negative hyperplanes. But before optimizing, how does sklearn first find the positive and negative support vectors and therefore the margin hyperplanes?
I apologize if this isn't a full answer but here's what I found so far:
It depends on what you're using in their api.
According to sklearn's svm documentation, they use the libsvm library for implementing their classifiers for SVC and NuSVC. For those cases, it is based on top of libsvm which is implemented using this paper. The paper is long so I won't summarize it but you should read it for further understanding. For other implementations, they have the CalibratedClassifierCV. You can read their source code for it here if you'd like their exact implementation. (Hint: Hold Ctrl and click a method to see its implementation. Some mice have a button to go back a page on their side if you'd like to back out too)
I hope this helps!
In the support vector machine (SVM) algorithm, the optimization goal is to find the hyperplane that maximally separates the positive and negative examples in the training data. This is known as the maximum margin hyperplane.
To find this hyperplane, the SVM algorithm first identifies the examples in the training data that are closest to the decision boundary (a conceptual line, surface, or hyperplane that separates the input space into different regions or classes), known as the support vectors. The hyperplane is then defined by the support vectors, and the distance between the hyperplane and the nearest support vectors is known as the margin. The optimization goal of the SVM is to maximize the margin, so that the hyperplane is as far away from the support vectors as possible.
In scikit-learn, the SVM algorithm uses a quadratic programming solver to find the maximum margin hyperplane. This solver iteratively improves the hyperplane by adjusting the weights and bias parameters of the model, with the goal of maximizing the margin between the positive and negative examples. The support vectors are identified during the optimization process, and the resulting hyperplane is used to make predictions on new data.