I am trying to detect anomalies through a column called count. The data is a time-series data and it is present for every 5 minutes for each day. The dataframe looks like this:

datetime         |  count
2021-03-31 00:05 |  25
2021-03-31 00:10 |  13
2021-03-31 00:15 |  0
2021-03-31 00:20 |  3
...              | ...
2021-04-15 22:10 | 111
2021-04-15 22:15 | 0
2021-04-15 22:20 | 9

However, the variance on the count column is huge, as a results of which when I am trying to use rolling z-score technique with a window of 288 (every 5 mins, means 12 data points in an hour, 24*12 = 288, 1 day window) and a threshold of -3, +3 as per central limit theorem, its failing because for most of the data points the scores are way outside these ranges as a result of which around 50% points are being declared as anomaly. Similarly I have already tried Isolation Forest, One Class SVM and Elleptic Envelope technique but nothing seems to be working. When I try to plot the anomalies, the graph gets filled with anomaly points, and the plot also doesn't make any sense.

I haven't worked with this kind of data earlier and I think I am going wrong somewhere. Can anyone suggest me what steps should I do to detect anomalies in this type of data and what might be some good algorithms which I can try?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.