# Tuning C hyper parameter in Soft Margin SVM in Matlab

How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?

The easiest way to tune a single hyperparameter is to use what is called the elbow method. Do the following:

• Define a range of C you want to try, i.e C = [1.0, 1.5, 2.0, ...]
• Loop over all values of C in your range
• Train a new model with the current value of C
• Evaluate each model on the validation set and store the results

If you didn't choose a too narrow range you should visually see your optimal value of C by finding the value that either minimize or maximize your metric.

If you need to tune C alongside other hyperparameters you can use strategies such as grid search, randomized search or bayesian search.

• I think you mean Validation Set. One should never use the test set for hyperparameter tuning! Apr 9 '19 at 21:29
• You are correct. Thank you for noticing! Edited and corrected. Apr 9 '19 at 21:35
• I am using 10 fold cross-validation with test and train partitions. so I don't have validation set. is it applicable to use it?
– gin
Apr 10 '19 at 8:12
• there is 'OptimizeHyperparameters' function in matlab that uses Bayesian Optimization to generate good parameters. Do you think it is suitable for the linear soft margin SVM ? mathworks.com/help/stats/…
– gin
Apr 10 '19 at 8:17
• Yes, it is still applicable but any time you want to perform tuning you need to set aside some validation data to do it on. This applies regardless of the method you use to tune. Apr 10 '19 at 8:36