4
$\begingroup$

I am using an unsupervised isolation forest algorithm and computing anomaly scores to detect outliers from a 2 dimensional toy dataset. From a scatter plot, I am able to detect/visualize the data points with the highest anomaly scores (example: top 10 or top 15 outliers from the data)as my outliers. Is the number of outliers subjective to user decision (for example:Anomaly score of 0.5>S>1 is an outlier and everything less than 0.5 is not) or is there a way to detect the optimal number of outliers based on the anomaly scores?

Outlier detection using Isolation Forest

$\endgroup$
1
  • 1
    $\begingroup$ Hello! Welcome to our community! Would you mind posting your plot, just to illustrate? Maybe we can identify a certain kind of distribution. For example in a Gaussian distribution we consider a sample to be outlier if it differs from the mean by $2\sigma$ $\endgroup$ – Pedro Henrique Monforte Apr 9 '19 at 19:01
1
$\begingroup$

The way Isolation Forests seem to be used in most of the cases involve having some kind of prior "guess" about what proportion of outliers you expect (if you want to be on the safe side, you might be encline to increase it for instance). Based on the proportion you set (it is the contamination parameter in scikit-learn), the observations are labeled according to their anomaly score.

That being said, nothing prevents you from coming up with your own rules after you ran the Isolation Forest. You can plot the anomaly scores and then see for yourself if another threshold could make sense based on the distribution of the scores. This would be a more empirical rule.

$\endgroup$
1
  • $\begingroup$ Thanks for your answer. May I ask you kindly to have a look at related post here? $\endgroup$ – Mario Mar 16 at 12:33
0
$\begingroup$

This would be an empirical rule. you can come up with some logic to label outlier(if something 2-std away from the mean, label as an outlier then use this percentage to set contamination level).

$\endgroup$
0
$\begingroup$

There is an attribute called as score you can use it as model.predict_score(X) which returns anomaly score based on this you can classify it as inlier or outlier.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.