Looking for a good package for anomaly detection in time series

Question

Is there a comprehensive open source package (preferably in python or R) that can be used for anomaly detection in time series?

There is a one class SVM package in scikit-learn but it is not for the time series data. I’m looking for more sophisticated packages that, for example, use Bayesian networks for anomaly detection.

Answer 1

I know I'm bit late here, but yes there is a package for anomaly detection along with outlier combination-frameworks.

The package is in Python and its name is pyod. It is published in JMLR.

It has multiple algorithms for following individual approaches:

1. Linear Models for Outlier Detection (PCA,vMCD,vOne-Class, and SVM)
2. Proximity-Based Outlier Detection Models (LOF, CBLOF, HBOS, KNN, AverageKNN, and MedianKNN)
3. Probabilistic Models for Outlier Detection (ABOD and FastABOD)
4. Outlier Ensembles and Combination Frameworks(IsolationForest and FeatureBagging)
5. Neural Networks and Deep Learning Models (Auto-encoder with fully connected Neural Network)

Finally, if you're looking specifically for time-series per se, then this github link will be useful.

It has the following list packages for timeseries outlier detection:

datastream.io

skyline

banpei

AnomalyDetection

Answer 2

There are multiple ways to handle time series abnormalities-

1. If abnormalities are known, build a classification model. Use this model to detect same type of abnormalities for time series data.

2. If abnormalities are unknown, what we have done in our organization is a combination of clustering and classification.

First use LOF/K-means/Cook's distance to identify outliers. Convert entire data into classification problem as we have got 2 classes now- Outliers and normals. Now build a classification model and get rules (classification model ) to identify abnormality at run time (time series data).

3. If abnormalities are unknown, during my research, most common way of identifying abnormalities is to build a normal model and any deviation from normal model (error) is abnormal, so in your case, you forecast your time series for next hour and then compare with actual values. If error is more than expected, something abnormal is happening.

I was not able to find any direct package in Python or R to do so, as nobody knows what is really abnormal, in all the cases it has been related to outlier detection.

This & this may be useful to you.

Answer 3

Try Prophet Library

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

More at: Anomaly detection in time series with Prophet library

• https://towardsdatascience.com/anomaly-detection-with-time-series-forecasting-c34c6d04b24a

Answer 4

I found Matrix profiles useful, especially as they do not require much knowledge up ahead - not even the duration of the time frame. For what I understood they basically rely on autocorrelation. The program can be used to feed in some data and find the anomalies, I have not used it for prediction yet.

https://matrixprofile.docs.matrixprofile.org/