2
$\begingroup$

What are general methods for outlier detection that do not assume any underlying distribution in the data? I have a dataset with the prizes of the rents in London, as well as their location, number of bedrooms, living rooms and bathrooms. I want to identify outliers in this data, where some of the variables are discrete and some of them are continuous. Any ideas on how to do this?

$\endgroup$
  • $\begingroup$ are you familiar with influential points? $\endgroup$ – Media Apr 19 '18 at 11:39
  • $\begingroup$ Yes, but only in the context of linear models $\endgroup$ – David Masip Apr 19 '18 at 11:41
  • $\begingroup$ They are probable outliers. $\endgroup$ – Media Apr 19 '18 at 12:10
2
$\begingroup$

Dbscan seems a great choice for you, look at scikit-learn implementation for further http://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html discovery.

About being discrete or continuous, it actually doesn't matter, what you have to look at it is if the scale is the best suited for the algorithm in hand (and scikit-learn has algorithms to handle that).

Other tip is to actually see if the attributes fit on a distribution, some of them might and parametric methods of detecting outliers are better suited for the task.

$\endgroup$
3
$\begingroup$

You can use proximity-based outlier detection methods. These methods fall under 2 different categories, 1) Cluster-based, and 2) Distance-based.

Cluster-based methods will define a point as an outlier if its distance to cluster centroids is large. If you have both categorical and continuous variables, then it is important to ensure that the clustering method you use is appropriate for mixed data-types. Clustering methods appropriate for both types include k-Prototypes or Squeezer.

To use a distance-based method, you could use the k-nearest neighbor algorithm, and define an outlier as a data point whose distance from its k neighbors is much larger than normal data points.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.