There are several anomaly detection techniques, such as statistical, density-based, depth-based, clustering, etc.
Given a dataset, what are the criteria or how should I choose which one of the techniques above (not the algorithms inside the techniques). In general, why choose a particular technique.
It is not a bad idea to try more than one methods. Then see where they agree and were they disagree. Try to explain the differences. Do some statistical analyses and visualizations.
It is also important to approach the issue from a practical perspective. What is the reason you want to identify outliers? How outliers will affect your key performance metrics?
Also remember that the very notion of an outlier can be relative. What is an outlier for a linear regression of order 1 may not be for a linear regression of order 2 (containing quadratic terms). What is an outlier from a Normal PDF point of view is not from an Exponential PDF point of view.
You first need to specify if you are going to work on anomaly detection or novelty detection.
The difference between them is that in anomaly detection, your data contains anomalies, but in novelty detection, your data contains only valid data.
After that you need to choose the algorithms depending on what you have chosen before, there are some algorithms that work only with one type, and there are algorithms that work with both types.
