Anomaly detection on time series

Question

I've just started working on an anomaly detection development in Python.

My data sets are a collection of timeseries. More in details, data are coming from some sensors/meters which record and collect data on boilers or other equipments.

As I said before, the data which I have to work with, are timeseries, so a timestamp and the relative value detected by sensor; a value is anomalous when it's bigger or smaller than the others near it; basically a peak.

I need to develop an unsupervised classification model, because I haven't labels for all data.

Another important aspect, is that this data are "season dependent"; in fact a boiler should be has a higher consumptions in winter than summer. Those values must not be considered as anomalies.

Since I've no experince on this topic, I'm here to ask you, what is the best algorithm/approace to solve this problem.

Furthermore, do you know some books or links to suggest?

Answer 1

For understanding the seasonality of time series data I would start with Holt-Winters Method or ARIMA. Understanding these algorithms will help with understand how time series forecasting works. Time series forecasting

For unsupervised classification, I would start with something like k-means clustering for anomaly detection.
Anomaly Detection with K-Means Clustering

These links should be a good starting point, I hope this helps.

Answer 2

You can have a look here, where many open-source algorithms specifically for anomaly detection on time-series data (e.g. metrics) are collected, both for online of offline settings.

Almost all of them are unsupervised approaches that require no labels to detect the anomalies.

They also automatically handle some of the issues you mentioned, like seasonality.

Answer 3

We can use Local Outlier Factor (LOF) or Random Forest along with a month column. Train these algorithms with your timeseries data (the actual values column) plus on month column to handle the seasonality.

Moreover we can use Auto-Encoders to detect anomalies in Robust ways.

The last method I can suggest is STL (Seasonality, Trend and Losses) decomposition and using anomaly detection algorithm on the residuals (data after removing seasonality and trend component)

Answer 4

About anomalies detection, you have a bunch of methods. In the jargon they are called outliers, and Wikipedia's Outlier article is a very good start.

Most answers from Time Series will advise to use an Exponential smoothing (in the Holt-Winters version to take care of the seasonality), or the *ARIMA (of which Exponential smoothing is a individual case).

This may be good. However, for situation like this, Engineers are used to filter the sensor with a Kalman filter. The mathematical step is steeper, though.