2
$\begingroup$

I am currently using an one class classification svm and I am trying to boost the classification results by employing more than one svm-occ with varying gamma parameters and combine these decisions with majority voting. But the results are not that satisfactory , since with a careful choice of gamma parameter I can get better results with just one-class svm. So , I was just wondering if I can employ also another one-class classifier e.g. SVDD and then combine its decisons with svm-occ . But the problem is my project is being developed in python enviroment and I can't find an implementation of SVDD with sklearn. So is there any suggestions which classifiers to use and if an according python implementation exists or how to combine them to obtain in overall better classification results ? (**I am keeping the nu parameter on svm-occ fixed)

$\endgroup$
0
$\begingroup$

Ummm... SVM is a linear model, so if you average the predictions of several models with different regularization, what you get is the same result of another model which uses a different regularization.

Regarding the intent behind your question: there's more methods than SVM and SVDD. You can check the PyOD package for ensemble models for outlier detection.

$\endgroup$
1
  • $\begingroup$ Yeah you are right , your first answer is very insightful! But I am using the average of predictions of several models as a tuning procedure, cause I am dealing with novelty detection, not outlier detection, and I don't have any knowledge in advance of the other class data and therefore can't implement tuning. PyOD package is about outlier detection, not novelty and python sklearn only posseses SVM one class and Local Outlier Method for novelty detection. $\endgroup$
    – user110508
    Jan 22 at 23:12
0
$\begingroup$

SVM is effectively a 2-layer NN. It is better to use a neural network for creating an embedding. Since, you don't have negative examples, you can't use something like a Siamese Network. One good way for you to create those embeddings would be use the bottleneck layer of an autoencoder. Or if you have image data, use ResNet to get the embeddings (the penultimate layer). Or if you have tabular data, use TabNet to get the embeddings.

Once you have the embeddings, take the distance of the embeddings (or you can use K-Means to get cluster heads and then take distance from them) from the new example. You set a threshold, it the distance is larger than that. Then, yay, it is a novelty. The distance you can use is L1, L2 or cosine.

For all the above techniques, the code is readily available in Python.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy