Why is Local Outlier Factor classified as Unsupervised if it requires training data with no outliers?

Question

In Scikit-Learn, the Local Outlier Factor (LOF) algorithm is defined as an unsupervised anomaly detection method.

So then I don't understand why this algorithm requires pre-filtered training data. Perhaps "training data" here simply means "data to start with?" But the example code provided by SciKit-Learn clearly shows training data which explicitly contains NO anomalies. Does that mean that this model would NOT work if the training data contains anomalies? And more importantly, how do I find anomalies in the training data using this algorithm?

Here is the website. https://scikit-learn.org/stable/auto_examples/neighbors/plot_lof_novelty_detection.html#sphx-glr-auto-examples-neighbors-plot-lof-novelty-detection-py

I changed the sample code to include anomaly data in the training data set and the model still found a decision boundary that looked correct. So am I just getting confused because of the way the documentation and sample code is written? Or is does this model really need a clean training data set?

Answer 1

It appears that SciKit-Learn implements two "modes" for LocalOutlierFactor, where one is unsupervised and one is semi-supervised. I think I misunderstood the documentation and implementation at first.

The two modes are "outlier" and "novelty". The sample code provided on the SciKit-Learn website uses Novelty detection mode. When using the "outlier" mode the model does NOT require a training dataset--an unlabeled dataset is used as whole and a score is given to every data point.

novelty : boolean, default False By default, LocalOutlierFactor is only meant to be used for outlier detection (novelty=False). Set novelty to True if you want to use LocalOutlierFactor for novelty detection. In this case be aware that that you should only use predict, decision_function and score_samples on new unseen data and not on the training set.

Answer 2

If the training data already contains outliers there is a chance that a new outlier will not be detected because it can be in the neighborhood of an outlier in the training data.

For example, consider the possible case where n_neighbors=1 and the closest neighbor to a new outliers is an "old outlier", then it couldn't be detected.

That doesn't mean, however, that the algorithm can't detect some new outliers (it probably will, as you saw with your data, if the number of outliers in your training data is relatively small).

Answer 3

As you already mentioned, there is a difference in the LOF library between 'normal' outlier detection and novelty detection.

In a simple terms, the outlier detection finds outliers within itself (the whole training dataset) like many other outlier detection algorithms. On the other hand, what the novelty detection allows you to do is to compare the density of a given test data point vs the density of the closest neighbor train data point. Essentially, here you can compare train vs. test or just generally two different datasets and identify the areas where there are outliers/no outliers.