18
$\begingroup$

Is there a comprehensive open source package (preferably in python or R) that can be used for anomaly detection in time series?

There is a one class SVM package in scikit-learn but it is not for time series data. I’m looking for more sophisticated packages that, for example, use Bayesian networks for anomaly detection.

$\endgroup$
  • $\begingroup$ we need to write one, bro!! :P $\endgroup$ – Arpit Sisodia Jun 15 '18 at 13:22
21
$\begingroup$

I know I'm bit late here, but yes there is a package for anomaly detection along with outlier combination-frameworks.

Package is in python language and name of package is pyod (https://github.com/yzhao062/Pyod). It is published in JMLR.

It has multiple algorithms for following individual approaches:

  1. Linear Models for Outlier Detection (PCA,vMCD,vOne-Class, and SVM)
  2. Proximity-Based Outlier Detection Models (LOF, CBLOF, HBOS, KNN, AverageKNN, and MedianKNN)
  3. Probabilistic Models for Outlier Detection (ABOD and FastABOD)
  4. Outlier Ensembles and Combination Frameworks(IsolationForest and FeatureBagging)
  5. Neural Networks and Deep Learning Models (Auto-encoder with fully connected Neural Network)

Finally, if you're looking specifically for time-series per se, then this github link will be useful.

It has the following list packages for timeseries outlier detection:

datastream.io

skyline

banpei

AnomalyDetection

| improve this answer | |
$\endgroup$
  • $\begingroup$ any way to work on context based abnormalities? $\endgroup$ – Arpit Sisodia Aug 9 '19 at 10:19
  • $\begingroup$ are you talking about anomaly detection or outlier detection? There is a difference. $\endgroup$ – Arpit Sisodia Aug 9 '19 at 10:21
6
$\begingroup$

There are multiple way to handle time series abnormalities-

1) If abnormalities are known, build a classification model. Use this model to detect same type of abnormalities for time series data.

2) If abnormalities are unknown, What we have done in our organisation- is a combination of clustering and classification-

First use LOF/K-means/Cook's distance to identify outliers. Convert entire data into classification problem as we have got 2 classes now- Outliers and normals. Now build a classification model and get rules( classification model ) to identify abnormality at run time ( time series data).

3) If abnormalities are unknown, During my research, most common way of identify abnormalities is to build a normal model and any deviation from normal model ( error ) is abnormal, so in ur case you forecast your time series for next hour and then compare with actual values. If error is more than expected , something abnormal is happening.

I was not able to find any direct package in python or R to do so, as nobody known what is really abnormal :P, in all the cases it has been related to outlier detection.

some useful links-

https://machinelearningstories.blogspot.com/2018/12/easiest-way-of-detection-abnormality.html

https://machinelearningstories.blogspot.com/2018/07/anomaly-detection-anomaly-detection-by.html

| improve this answer | |
$\endgroup$
2
$\begingroup$

Try Prophet Library

Prophet is a procedure for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works best with time series that have strong seasonal effects and several seasons of historical data. Prophet is robust to missing data and shifts in the trend, and typically handles outliers well.

More at: Anomaly detection in time series with Prophet library

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.