I have a problem where I am trying to apply a one-class svm to detect outliers. I am training on a dataset of true cases using a one-class radial svm and then predicting for both false and true cases. It is worth noting that the true cases I am training on do contain a proportion of misclassified cases. This is the nature of the problem and not something that can be corrected for or identified easily.
I have applied hyperparameter tuning through a grid search with cross-validation to get the best values of nu and gamma for the model. However, when I do this and predict on all cases, I get poor separability between true and false cases based on the resulting decision values. I actually get far better separability from using a fixed value for C and gamma of 1 and 0.05 respectively.
My intuition is that this lack of separability is down to the misclassified cases in the true training data. As such, my question is should hyperparameter tuning be applied to a one-class svm on data with misclassified values? If so, should nu and gamma both be tuned? Any papers discussing this problem would be great as I haven't found any resources relating to it.
I am aware that nu is related to C by nu = A+B/C where A, B are constants that are difficult to calculate and that
The parameter nu is an upper bound on the fraction of margin errors and a lower bound of the fraction of support vectors relative to the total number of training examples
So nu of 0.01 would mean at most 1% of your training examples would be misclassified. A good discussion on nu is here. I am searching values 0.0001 0.0010 0.0100 0.0500 0.1000 0.1500 0.2000 0.2500 0.3000 0.3500 0.4000 0.4500 0.5000.
I am also aware that
Gamma is a parameter of the RBF kernel and can be thought of as the ‘spread’ of the kernel and therefore the decision region. When gamma is low, the ‘curve’ of the decision boundary is very low and thus the decision region is very broad. When gamma is high, the ‘curve’ of the decision boundary is high, which creates islands of decision-boundaries around data points.
A good post on gamma with intuitive visualisations is here. I am searching across gamma values of 1x10^-04 1x10^-03 1x10^-02 1x10^-01 1x10^+00 1x10^+01 1x10^+02 1x10^+03 1x10^+04 1x10^+05
So should I hold nu constant at a high value to assume a high number of misclassified cases? Also should I search across different values of gamma?
