I have time-series data with alerts (every minute) that I need to find anomalies in.
I am looking for a library which can do unsupervised learning of this data and detect anomalies in the data.
Which would be the best Python library that can handle this scenario? I have been reading upon all of the these (pyod vs pycaret vs prophet vs scipy vs matrixprofile) but could not zero-in on the best one.
From a time-series point of view, an anomaly could be defined a value that doesn't fit most known patterns.
Now, we should define the time range and the method to detect the anomalies, and it depends on the data you are using.
For instance, if you have a very simple business case to detect abnormal peaks in a short time range, you could merely set the average value from the 10 last minutes and set a threshold (ex: +50% of the average value) that would detect an anomaly.
For simple shapes' detection in short-term time series, you could apply first (and potentially second-order) derivates to detect simple variations and also apply a threshold.
But if you want to detect complex shapes in longer time ranges, you can apply unsupervised time-series clustering but it is only applicable if your date has some cyclic behavior (otherwise you couldn't detect the start and the end of each time frame).
Some non-linear algorithms like PacMAP or UMAP could generate very clear clusters and detect outliers easily.
https://www.kaggle.com/code/frankmollard/pca-t-sne-umap-trimap-pacmap/notebook
It is not possible to know what is the best model, method or library with so little information on the problem. The nature of anomalies, the performance targets, the computational requirements, the need for explainability, the amount of data available, et.c. are all undefined. But even with all these things, it may not be possible to know the optimal one up-front. Such is the nature of many data science problems, because they are task and data dependent, in hard to predict ways.
So, start with the simplest thing that could possibly work, and move up from there!
