2
$\begingroup$

I've been using isolation forest for anomaly detection, and reviewing its parameters at scikit-learn (link). Looking at "bootstrap", I'm not quite clear what using bootstrap would cause. For supervised learning, this should reduce overfitting, but I'm not clear what the effect on anomaly detection should be.

I think it would require the trees to achieve more "consensus" about what the anomaly is, therefore, reducing the effect of any single feature. I.e, an anomalous observation would probably need to be anomalous consistently and over a number of features (?).

Is this a correct interpretation of this parameter?

Improve this question
$\endgroup$
2
  • $\begingroup$ Your argument seems correct for the effect of a smaller max_features, but not really for bootstrap. (I can't think of a particularly clear effect for bootstrap, except that it effectively decreases the sample size for a given max_samples.) $\endgroup$
    – Ben Reiniger ♦
    May 13, 2021 at 14:18
  • 1
    $\begingroup$ I was mistaken in how bootstrap is used. @Julio's answer cleared that up for me. $\endgroup$
    – Ruslan
    May 13, 2021 at 17:53

1 Answer 1

Reset to default
2
$\begingroup$

This is well explained on the original paper Section 3.

As well as in the Supervised Random Forest, Isolation Forest makes use of sampling on both, features and instances, so the latter in this case helps alleviate 2 main problems:

  1. Swamping

Swamping refers to wrongly identifying normal instances as anomalies. When normal instances are too close to anomalies, the number of partitions required to separate anomalies increases – which makes it harder to distinguish anomalies from normal in- stances.

  1. Masking

Masking is the existence of too many anomalies concealing their own presence.

Contrary to existing methods where large sampling size is more desirable, isolation method works best when the sampling size is kept small. Large sampling size reduces iForest’s ability to isolate anomalies as normal instances can interfere with the isolation process and therefore reduces its ability to clearly isolate anomalies. Thus, sub-sampling provides a favourable environment for iForest to work well. Throughout this paper, sub-sampling is conducted by ran- dom selection of instances without replacement.

enter image description here

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the great explanation and reference. It would seem that sub-sampling would tend to increase the number of samples identified as anomalies $\endgroup$
    – Ruslan
    May 13, 2021 at 6:56
  • $\begingroup$ This answers why subsampling (especially, to a smaller subset than would be usual for random forests) helps, but doesn't address whether the subsamples are drawn with replacement (bootstrapping) or not. $\endgroup$
    – Ben Reiniger ♦
    May 13, 2021 at 14:16
  • $\begingroup$ Python's iForest documentation states: "bootstrap: If True, individual trees are fit on random subsets of the training data sampled with replacement. If False, sampling without replacement is performed." $\endgroup$
    – Ruslan
    May 13, 2021 at 18:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.