1
$\begingroup$

I'm in a situation where I need to limit the number of support vectors (SVs) in my support vector machine binary classifier. A simple way to do this would be to manually remove the least important SVs. Does anyone know of a way to do this?

I've tried manually modifying the following fields of the SVM to remove SVs, but without success:

clf.dual_coef_
clf.intercept_
clf.n_support_
clf.support_
clf.support_vectors_
clf.shape_fit_

where clf is my RBF SVM.

Note that increasing regularization (by decreasing C) does not necessarily reduce the number of SVs (https://dgroppe.com/2018/01/21/increased-regularization-does-not-necessarily-decrease-the-number-of-support-vectors/)

$\endgroup$
1
$\begingroup$

I was having the same problem. I managed to do it with Scikit's OneClassSVM; maybe it can also help in your case. I adjusted some of the parameters you mentioned plus a few more (just in case) including:

  • _dual_coef_
  • dual_coef_
  • _intercept_
  • intercept_
  • support_
  • support_vectors_

Ex: For OneClassSVM, if you obtained 10 support vectors (after fitting dataset X) and wanted to prune the last one (index -1) the code would be as follows:

model = OneClassSVM()
model.fit(X)

model._intercept_ = model._intercept_ + temp_model.dual_coef_[0][-1]
model.intercept_ = model.intercept_ + temp_model.dual_coef_[0][-1]
model._dual_coef_ = np.delete(model._dual_coef_, -1, axis=1)
model.dual_coef_ = np.delete(model.dual_coef_, -1, axis=1)
model.support_ = np.delete(model.support_, -1, axis=0)
model.support_vectors_ = np.delete(model.support_vectors_, -1, axis=0)
$\endgroup$
1
$\begingroup$

One option could be to use Nu-Support Vector Classification, implemented in SciKit Learn as NuSVC and written about in the user guide

This introduces a parameter Nu

An upper bound on the fraction of training errors and a lower bound of the fraction of support vectors. Should be in the interval (0, 1].

By reducing Nu you can reduce the number of Support Vectors

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.