I am working on an anomaly detection problem to detect fraud in insurance claims. I have used the PyOD package and used algorithms like ABOD, CBLOF, Isolation Forest, and AutoEncoder. I couldn't find any way to identify the important features which make the data points anomalies ( like variable Importance in Random Forest). Is there a way to identify the important features in unsupervised anomaly detection?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
There have been workshops dedicated to "outlier detection and description" (ODD), but there came out nothing from them that convinced me, unfortunately. but YMMV. It definitely won't be enough to just use some library! You'll need to read and implement papers. There are subspace outlier detectors and correlation outliers, for example, that will tell you which features were relevant for a particular outlier.
But in general I believe you'll quickly run into multiple testing problems: in real data, every point is anomalous if you just try attribute combinations hard enough. Donald Trump is anomalous because of his orange skin, fake hair, and small hands, for example. If you only look at his xenophobia, he probably is pretty normal, unfortunately.
answered Mar 19, 2019 at 19:35
$\begingroup$
$\endgroup$
0
This question has been asked so many times, yet I believe no widely accepted answer exists, especially in the case of black box models such as neural networks.
A way to go may be sensitivity analysis, i.e. evaluate the change in the output of the model for small changes in the individual inputs. The higher the change in the output, the more important the feature.
