5
$\begingroup$

I'm working on an unsupervised anomaly detection task on time series using isolation forest algorithm.
I'm developing in Python, more in detail using sklearn.

I found out a lot of examples on this, but what is not very clear, is how to set the contamination param during the instantion of IsolationForest.

Looking the documentation, contamination is

the amount of contamination of the data set, i.e. the proportion of outliers in the data set.

Should I use some statistical techniques to identify this percentage?

Thank you

$\endgroup$
1
$\begingroup$

I think the answer is "it depends". The contamination parameter simply controls the threshold for the decision function when a scored data point should be considered an outlier. It has no impact on the model itself.

Depending on your application it could make sense to use some statistical analysis to get a rough estimate of the contamination. (If you could narrow it down more specifically then you wouldn't need the isolation forest...)

Usually, I would say, this is informed by some business expectation. Something like "we expect to see a similar rate to our competitors who have reported a rate of x".

Or you expect a certain number of outliers in your dataset. Then you can use the raw scores to find a threshold that gives you that number and set the contamination parameter retrospectively when applying the model to new data.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the answer! can you please give me some examples of statistical analysis that i can apply? thanks $\endgroup$ – Giordano Jul 4 '19 at 21:42
  • 1
    $\begingroup$ I'd probably need to know more about your data. Maybe ask a new question? :) $\endgroup$ – oW_ Jul 8 '19 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.