How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?


1 Answer 1


The easiest way to tune a single hyperparameter is to use what is called the elbow method. Do the following:

  • Define a range of C you want to try, i.e C = [1.0, 1.5, 2.0, ...]
  • Loop over all values of C in your range
    • Train a new model with the current value of C
    • Evaluate each model on the validation set and store the results
  • Plot your metric over over your range of C's

If you didn't choose a too narrow range you should visually see your optimal value of C by finding the value that either minimize or maximize your metric.

enter image description here

If you need to tune C alongside other hyperparameters you can use strategies such as grid search, randomized search or bayesian search.

  • 1
    $\begingroup$ I think you mean Validation Set. One should never use the test set for hyperparameter tuning! $\endgroup$ Apr 9, 2019 at 21:29
  • 1
    $\begingroup$ You are correct. Thank you for noticing! Edited and corrected. $\endgroup$ Apr 9, 2019 at 21:35
  • $\begingroup$ I am using 10 fold cross-validation with test and train partitions. so I don't have validation set. is it applicable to use it? $\endgroup$
    – gin
    Apr 10, 2019 at 8:12
  • 1
    $\begingroup$ there is 'OptimizeHyperparameters' function in matlab that uses Bayesian Optimization to generate good parameters. Do you think it is suitable for the linear soft margin SVM ? mathworks.com/help/stats/… $\endgroup$
    – gin
    Apr 10, 2019 at 8:17
  • $\begingroup$ Yes, it is still applicable but any time you want to perform tuning you need to set aside some validation data to do it on. This applies regardless of the method you use to tune. $\endgroup$ Apr 10, 2019 at 8:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.