How to tune the C 'BoxConstraint' hyperparameter in soft margin SVM to get the best optimal value?
The easiest way to tune a single hyperparameter is to use what is called the elbow method. Do the following:
- Define a range of C you want to try, i.e
C = [1.0, 1.5, 2.0, ...]
- Loop over all values of C in your range
- Train a new model with the current value of C
- Evaluate each model on the validation set and store the results
- Plot your metric over over your range of C's
If you didn't choose a too narrow range you should visually see your optimal value of C by finding the value that either minimize or maximize your metric.
If you need to tune C alongside other hyperparameters you can use strategies such as grid search, randomized search or bayesian search.