Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data.
What should be the Reconstruction error threshold I should give to classify it as whether it is anomalous or not?
I have currently set it to 0.1 , but it doesnt find any anomalies(actually test data have lot of anomalies) What are the factors that I should consider before setting the value?
If one considers prediction of anomalous status as binary classification (i.e., if reconstruction error < threshold, classify as normal, else classify as anomalous), one can find the threshold that maximizes some appropriate metric of classification performance (e.g., F-beta) by optimizing the metric over a suitable validation set containing normal and anomalous data. See Malhotra et al., 2016 for an example of how to do this for time series.
