1
$\begingroup$

I read this article from sklearn: https://scikit-learn.org/stable/modules/outlier_detection.html

While these algorithms are very useful for outlier detection, I'm surprised to see that they are not using labels to find outliers.. Is it normal? Would it be not useful to use them?

Outlier detection is then also known as unsupervised anomaly detection

Are there existing libraries which provide supervised outlier detections algorithms?

More explanations:

How to know if a data point is really an outlier without the label? Let say we want to predict the price of a house. An house having a lot of good features (huge size, big swimming pool, big garden, .) might be an outlier if the price is low, or as a normal point if the price is higher than average.

$\endgroup$
2
  • 1
    $\begingroup$ About your housing example: the price is a label if the task is to predict the task. For another task (such as outlier detection) it is just another feature. If the task was to predict the size of a house from its marker price, then the label is the size and the price is the sole feature. Labels and features are defined from the task and the available data. $\endgroup$ – Learning is a mess Nov 14 '19 at 10:37
  • $\begingroup$ True .. but what if we want outliers related to this task ? $\endgroup$ – nolw38 Nov 14 '19 at 10:51
2
$\begingroup$

Outlier detection can be performed in unsupervised fashion if there are no labels, or supervised fashion if there is a training set where outliers have already been marked as such by an "expert".

Unsupervised outlier detection is often (always?) based on density. Algorithms will call observations outliers if they are too far away from most of the other observations. This is useful to detect cases which are likely to be either errors or exceptional cases; and eventually, for instance, to remove the errors from the dataset.

Supervised outlier detection is less common. Indeed, in real world applications, it is rather easy to determine what should be the normal situations (e.g. some equipment mode of operation), but quite complex to define abnormal situations. You could have examples of abnormal situations, but probably not all possible abnormal situations. Then, a supervised learning outlier detection (basically a binary classification problem with normal and abnormal classes) will learn how to detect known abnormal situations, but will likely fail on unknown ones.

This is why there is the semi-supervised outlier detection (called novelty detection in the scikit-learn example). The training dataset contains only normal points, the model is trained to define the boundaries of the normal domain. Then, at classification step, the algorithm can predict if an observation can be deemed normal or not. Semi-supervised outlier detection techniques are often based on the likelihood that observations were generated by the same unknown process which generated the learning dataset (see Bayesian networks or Gaussian mixture models for instance).


Notes:

  • Unsupervised and semi-supervised techniques may be combined. For instance, you could use a first unsupervised approach to remove supposed outliers, then train a semi-supervised model on the remaining observations. The resulting model will likely be more robust than if outliers had been labelled as such, and a supervised model had been trained on the full dataset.
  • For semi-supervised or unsupervised outlier detection, the decision boundary position depends on model hyperparameters (expected fraction of outliers, distance to inliers, acceptable likelihood, etc.)
$\endgroup$
4
  • 1
    $\begingroup$ Good answer, but just missing the "Why"! I would add the "Why", something like "Intrinsic nature of outlier is such that it often (not always) occurs without prior knowledge (element of surprise), meaning in reality we wouldn't know about them beforehand thus no labels. This is what is shown in sklearn, although most of the cases labels are there BUT not being used, a bit confusing, but it is more plotting purposes to validate the approaches afterwards.!" The rest is well-explained in your answer. $\endgroup$ – TwinPenguins Nov 14 '19 at 6:24
  • $\begingroup$ I added an additional explanation on my question to make my point more understandable. Thank you for these clear explanations though $\endgroup$ – nolw38 Nov 14 '19 at 6:58
  • $\begingroup$ This clear breakdown is really useful @romain-reboulleau For deeper birds view on outlier detection I suggest having a look at robots.ox.ac.uk/~davidc/pubs/NDreview2014.pdf $\endgroup$ – Learning is a mess Nov 14 '19 at 10:39
  • 1
    $\begingroup$ @TwinPenguins Thanks for your comment, you are right, I improved my answer with that aspect (and also added a few details) $\endgroup$ – Romain Reboulleau Nov 14 '19 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.