1
$\begingroup$

I am currently exploring anomaly detection methods for my work and, basically I have gone through Local Oulier Factor and Isolation Forests, both unsupervised methods.

Now, the thing is, there might be a chance that I do not want a point that is far away to considered as an outlier, and so I would need some sort of supervised or semi supervised method for the outlier detection.

So what I am thinking is:

1.Label a bunch of points as outlier using LOF/IF.

2.Train a classifier on top of the labels, and then make manual adjustements if needed.

Is this what is considered a semi-supervised method? Does anybody have any experience with this sort of problem that could say if I am missing something here?

Also, because I am labeling outliers the dataset will be very unbalanced. My idea is to use bagging for this. Let's say my dataset is 1% outliers, I would train 100 equally proportional models (the outliers parts remains the same on each model, but the normal points change until I go over the entirety of the dataset) and then the final prediction is a vote of all the models. Is this stupid or a good idea?

Improve this question
$\endgroup$
5
  • $\begingroup$ Probably an ensemble of just 10 different folds will already give most of the benefit $\endgroup$
    – Jon Nordby
    Jun 14, 2020 at 10:23
  • $\begingroup$ Have you checked that the problem you are trying to fix (certain rare points being unwanted as anomalies) actually occurs? $\endgroup$
    – Jon Nordby
    Jun 14, 2020 at 10:24
  • $\begingroup$ Do you have a labeled validation and test-set already (manual/high quality), so you can evaluate performance on that? If not, that is the first thing I would do. $\endgroup$
    – Jon Nordby
    Jun 14, 2020 at 10:26
  • $\begingroup$ So we do not have any data yet. I was asked to look into the methods for anomaly detection and that is what I am doing. We do not have the labelled sets. I was thinking about creating the labels initially using unsupervised methods, and then manually correct them if necessary. My question was that, how would I go about training this very unbalanced Dataset (I know there is stuff like SMOTE and wtv but I wanted to try something simpler), and if there are methods that already do the unsupervised and supervised part together or if I need two algorithms for that. $\endgroup$
    – user98969
    Jun 14, 2020 at 11:30
  • $\begingroup$ Semi-supervised and Supervised Anomaly Detection are being actively researched, with plenty of papers out there. Plain old unsupervised is still the norm though $\endgroup$
    – Jon Nordby
    Jun 14, 2020 at 11:37

1 Answer 1

Reset to default
0
$\begingroup$

If you use the anomaly detector to label the data directly, there is no way the supervised step that follows can be better than that. One can of course go in an "adjust" the labels afterwards, but there is risk of being biased by the pre-existing labels if a human sees it up front.

Instead of sampling data to label randomly, you could sample weighted based on the anomaly score. This has two effects 1) reduces class imbalance 2) focuses labeling effort on likely anomalies

This assumes a well-tuned anomaly detector though. Which there is no good way to do without a validation set. That dataset should preferably be sampled randomly, to avoid bias.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.