I recently started playing with SVMs for a one class classification, I was able to get some reasonable classifications from real data and but was trying to optimize the nu and gamma parameters when I came across this example:

In the code below, I train an SVM with an array of ones, then I present the same array of ones for classification and it classifies all ones as outliers.

import pandas as pd
from sklearn import svm
import numpy as np

nu = 0.01
gamma = 1
ones = pd.DataFrame(np.ones(100))
clf = svm.OneClassSVM(nu=nu, kernel="rbf", gamma=gamma)
ones["predicted"] = clf.predict(ones)
#Returns -1 for all entries

My question is: why does this happen? I thought this data would be trivial for any parameter configuration.


1 Answer 1


What you are facing is a small but crucial definition difference:

novelty detection:

The training data is not polluted by outliers, and we are interested in detecting anomalies in new observations.

outlier detection:

The training data contains outliers, and we need to fit the central mode of the training data, ignoring the deviant observations.

OneClassSVM is an Unsupervised Outlier Detection. Therefor your data needs to have outliers in order for the algortihm to detect them. My best guess, why its prediction every input as an outlier is, that if there are no real outliers, everything must be an outlier.

Let me demonstrate this quickly. I adjustet the kernel to linear

import pandas as pd
from sklearn import svm
import numpy as np

nu = 0.5
gamma = 1.0
ones= pd.DataFrame(np.ones(100))

clf = svm.OneClassSVM(nu=nu, kernel="linear", gamma=gamma)
clf.predict(-1) # -1
clf.predict(1) # -1
clf.predict(1.00001) # 1
clf.predict(2) # 1
clf.predict(10) # 1
  • $\begingroup$ I didn't know of this definition, and it makes sense, as nu is supposed to be an upper bound on the number of outliers in the training data (meaning there should be outliers in the training data), however, OneClassSVM is listed as a novelty detection algorithm on scikit's documentation scikit-learn.org/stable/modules/outlier_detection.html $\endgroup$ Commented Jan 24, 2018 at 10:01
  • $\begingroup$ Thats odd, because in the OneClassSVM documentation it is lasted as outlier detection: scikit-learn.org/stable/modules/generated/… $\endgroup$ Commented Jan 24, 2018 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.