Outlier detection with sklearn

Question

I've been reading the sklearn documentation on outlier detection, and even the examples provided by the documentation. Once I have fitted my dataset, all I can do is apply the predict function to the estimator in order to find outliers. However, I would like to get the probabilities that the point is an outlier. Can this be done in sklearn? Is there an R package to do it?

I don't even know if there is a theoretical foundation of the outlier detection methods used by sklearn that allows you to give probabilities. If not, what is the criterium that tells you what is an outlier and what is not? Does it consider probabilities or some kind of non-probabilistic scores?

Any piece of information will be appreciated.

Edit

I would like the outlier method to consider the multivariate distribution of the data. I think that univariate detection methods are rather poor.

Answer 1

One of the easiest and quick way would be to plot a 'boxplot' where the "dots" above the upper whisker(T shaoed line) of the boxplot denotes an outlier(refer to the attached image). I hope this was what you are searching for. Do let me know if otherwise, may be I can help.

Answer 2

A simple trick to do outlier detection is to use the output probability of your model. If you are using a neural network for instance, you can use a softmax output which will give you a probability for each labels: \begin{equation} p(y=y_i) = \frac{e^{W_i^Tx+b_i}}{\sum_j e^{W_j^Tx+b_j}} \end{equation}

If your model is accurate, for most points in your dataset, the probability should be peaked on the true label. However, if you have an outlier, then the model should get confused and then return probabilities that are more spread out over the labels. You could measure this spread by measuring the entropy of the output softmax probability distribution p(y): \begin{equation} H[p] = - \sum_y p(y)\log p(y), \end{equation} where $y$ takes one the different output categories (labels). Entropy is a measure of uncertainty, so if you have $H(p)=0$, the model is confident of the output, whereas if $H[p] = \log(N_c)$ ($N_c$ the number of labels), the model has no idea what to predict. Thus, using a threshold on the entropy (you might have to tune that threshold depending on your specific problem), you categorize a point as being an outlier or not. I think this may be a good starting point for high-dimensional data. For low-dimensional data, you can always do some density estimation and use the density as a threshold.