2
$\begingroup$

Recently I have been trying different Scikit-Learn anomaly detection clustering methods, like

  • DBSCAN
  • Isolation Forest.

Based on how many training data I use, how I tweak on the algorithms

Example in DBSCAN I play around this min_samples and eps distance.

My problem Now I am getting different results, however this is where the problem is coming. I don't know how many anomalies I am supposed to get, so this leads to me not being sure what is working best.

I have read a it into adding anomalies yourself into the dataset. Is this a thing? Meaning that I would know if the found anomalies from the algorithm are the same as the one that have been added to the dataset.

I might have gotten this wrong.

The other problem is that the column I want to check for anomalis has numbers that really differ in magnitude so I also wouldn't what numbers I should add as anomalies.

Would appreciate some help.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ This does not answer your question, but you could try kaggle.com/mlg-ulb/creditcardfraud, this an anomaly detection problem. Many solutions are posted online so you can compare your approach/results with what other people did. $\endgroup$
    – Akavall
    Commented Feb 21, 2020 at 2:25

1 Answer 1

Reset to default
2
$\begingroup$

Without labels for which data points are anomalies, you have no way to know how many anomalies you should get.

The closest thing to adding anomalies to the dataset is using synthetic data for anomaly detection. I have seen some practitioners doing this, and failing. When you create the synthetic data, it matches your inductive bias*. The real world very rarely matches that inductive bias*. Afterwards, you can easily find a clustering algorithm with the same inductive bias* which can flag your synthetic data as anomalies. However, when that algorithm is deployed it will fail miserably.

*in plain language, inductive bias is the assumptions you have of how the world should be.

Improve this answer
$\endgroup$
2
  • $\begingroup$ What would you recommend in this case? $\endgroup$
    – E199504
    Commented Feb 21, 2020 at 7:28
  • $\begingroup$ The best option is to manually label a small subset of data and use that to evaluate your clusters. However, there are a few reasons this will not work. It may be unclear to you which data is anomalies. Further, if the anomalies are very rare, it may require labelling too many data points until you've labelled enough anomalies to evaluate the clusters. The less good option is to use a heuristic. This option has the same issues of inductive bias. Common heuristics are average distance between clusters, average distance between a data point and the centroid of the cluster. $\endgroup$
    – Aaron Elliot
    Commented Feb 21, 2020 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.