6
$\begingroup$

For simplicity let's assume the feature space is the XY plane.

$\endgroup$
  • 2
    $\begingroup$ If you have such a question, you probably have an intuition on why using clustering for outlier detection would be a nice strategy. If you add such information to your post, it shall definitely increase the visibility and the interest from others in answering your question. $\endgroup$ – Rubens Dec 12 '14 at 23:43
5
$\begingroup$

A very robust clustering algorithm against outliers is PFCM from Bezdek http://www.comp.ita.br/~forster/CC-222/material/fuzzyclust/fuzzy01492404.pdf.

In this paper Bezdek proposes Possibilistic-Fuzzy-C-Means which is an improvement of the different variations of fuzzy posibilistic clustering. This algorithm is particularly good at detecting outliers and avoiding them to influence the clusterization. So using PFCM you could find which points are identified as outliers and at the same time have a very robust fuzzy clustering of your data.

$\endgroup$
3
$\begingroup$

If your Data points are dense and noise points are away from the dense region, you can try DBSCAN algorithm.

http://en.wikipedia.org/wiki/DBSCAN

enter image description here

Tweak its parameters until u get a best fit.

$\endgroup$
2
$\begingroup$

Gaussian mixture modeling can - if your data is nicely gaussian-like - be used for outlier detection. Points with a low density in every cluster are likely to be outliers.

Works well in idealistic scenarios.

$\endgroup$
2
$\begingroup$
  1. Apply your clustering algorithm
  2. Calculate distance from all data points to its assigned cluster
  3. Label the data points furthest from a center as an outlier

Randomly generating 100 data points from three gaussians, clustering them with k-means, and marking the 10 'furthest from a center' data points gave the following graph: enter image description here

see this notebook for the full example

The burden of solving what "distance" means will already have to be solved for you to run a clustering algorithm. It will still be up to you to pick off what distance means an outlier. In this example, I just picked the N most distant data point, though you'll probably want to pick any number of data points over a certain number of standard deviations from a center.

$\endgroup$
1
$\begingroup$

Perhaps you could cluster the items, then those items with the furthest distance from the midpoint of any cluster would be candidates for outliers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.