Is there a way to manually remove support vectors from a scikit-learn model?

Question

I'm in a situation where I need to limit the number of support vectors (SVs) in my support vector machine binary classifier. A simple way to do this would be to manually remove the least important SVs. Does anyone know of a way to do this?

I've tried manually modifying the following fields of the SVM to remove SVs, but without success:

clf.dual_coef_
clf.intercept_
clf.n_support_
clf.support_
clf.support_vectors_
clf.shape_fit_

where clf is my RBF SVM.

Note that increasing regularization (by decreasing C) does not necessarily reduce the number of SVs (https://dgroppe.com/2018/01/21/increased-regularization-does-not-necessarily-decrease-the-number-of-support-vectors/)

Answer 1

I was having the same problem. I managed to do it with Scikit's OneClassSVM; maybe it can also help in your case. I adjusted some of the parameters you mentioned plus a few more (just in case) including:

• _dual_coef_
• dual_coef_
• _intercept_
• intercept_
• support_
• support_vectors_

Ex: For OneClassSVM, if you obtained 10 support vectors (after fitting dataset X) and wanted to prune the last one (index -1) the code would be as follows:

model = OneClassSVM()
model.fit(X)

model._intercept_ = model._intercept_ + temp_model.dual_coef_[0][-1]
model.intercept_ = model.intercept_ + temp_model.dual_coef_[0][-1]
model._dual_coef_ = np.delete(model._dual_coef_, -1, axis=1)
model.dual_coef_ = np.delete(model.dual_coef_, -1, axis=1)
model.support_ = np.delete(model.support_, -1, axis=0)
model.support_vectors_ = np.delete(model.support_vectors_, -1, axis=0)

Answer 2

One option could be to use Nu-Support Vector Classification, implemented in SciKit Learn as NuSVC and written about in the user guide

This introduces a parameter Nu

An upper bound on the fraction of training errors and a lower bound of the fraction of support vectors. Should be in the interval (0, 1].

By reducing Nu you can reduce the number of Support Vectors