# Why anomaly detection with IForest does not give expected results?

My goal is to develop an anomaly detection model with Isolation Forest in order to distinguish between normal and anomalous IPs by analyzing the web access logs and extracting some features in batches of 2 minutes. These features include how many times a specific IP did a request in the batch of 2 minutes, etc.

An example of training data is the following:

      0  1     2  3
0     4  2  37.5  1
1     1  1   0.0  0
2     2  1   0.0  0
3    39  1   0.0  0
4    34  1   0.0  0
5   158  1   0.0  0
6    15  1   0.0  0
7    61  1   0.0  0
8    13  1   0.0  0
9     1  1   0.0  0
10    1  1   0.0  0
11    2  2  75.0  1
12    8  1   0.0  0
13    1  1   0.0  0
14    1  1   0.0  0
15    1  1   0.0  0
16    1  1   0.0  0
17  274  1   0.0  0
18   40  1   0.0  0


An example of test data that I am using to predict is the following:

     0  1    2  3
0  392  1  0.0  0


Although I would expect this kind of input to be characterized as anomalous, since the first feature is larger than every correspondent feature in the training data, it is characterized as normal. My question is why something like that happens? I use contamination=0.1.

• Do you happen to have a higher value in the training dataset for the 1st feature value (392). As the training dataset you posted is just a sample dataset, we don't know for sure. Sep 7, 2022 at 7:31
• @Polymath No, this is the exact dataset that I used for this simple test. There does not exist higher number. Sep 7, 2022 at 7:33
• want to check one thing. Could you combine the test set with the training and then run the IF model, and see, if it identifies it as an Anamoly, contamination could be varied from Auto to .1 variations Sep 7, 2022 at 8:40
• @Polymath If I add the test data into the train data and then run the model, it identifies it as anomaly. This is weird. Why does this happen? Sep 7, 2022 at 14:49
• IF is an unsupervised learning algorithm, so there is "no training/ learning" taking place. It just creates the random decision trees with taking random features with thresholds to isolate data points. So, your test data point, having a higher value (392, for the first feature) would get isolated due to the threshold value Sep 7, 2022 at 14:59