General methods outlier detection

Question

What are general methods for outlier detection that do not assume any underlying distribution in the data? I have a dataset with the prizes of the rents in London, as well as their location, number of bedrooms, living rooms and bathrooms. I want to identify outliers in this data, where some of the variables are discrete and some of them are continuous. Any ideas on how to do this?

Answer 1

Dbscan seems a great choice for you, look at scikit-learn implementation for further discovery.

About being discrete or continuous, it actually doesn't matter, what you have to look at it is if the scale is the best suited for the algorithm in hand (and scikit-learn has algorithms to handle that).

Another tip is to actually see if the attributes fit on a distribution, some of them might, and parametric methods of detecting outliers are better suited for the task.

Answer 2

You can use proximity-based outlier detection methods. These methods fall under 2 different categories, 1) Cluster-based, and 2) Distance-based.

Cluster-based methods will define a point as an outlier if its distance to cluster centroids is large. If you have both categorical and continuous variables, then it is important to ensure that the clustering method you use is appropriate for mixed data-types. Clustering methods appropriate for both types include k-Prototypes or Squeezer.

To use a distance-based method, you could use the k-nearest neighbor algorithm, and define an outlier as a data point whose distance from its k neighbors is much larger than normal data points.