7
$\begingroup$

I've just started working on an anomaly detection development in Python.

My data sets are a collection of timeseries. More in details, data are coming from some sensors/meters which record and collect data on boilers or other equipments.

As I said before, the data which I have to work with, are timeseries, so a timestamp and the relative value detected by sensor; a value is anomalous when it's bigger or smaller than the others near it; basically a peak.

I need to develop an unsupervised classification model, because I haven't labels for all data.

Another important aspect, is that this data are "season dependent"; in fact a boiler should be has a higher consumptions in winter than summer. Those values must not be considered as anomalies.

Since I've no experince on this topic, I'm here to ask you, what is the best algorithm/approace to solve this problem.

Furthermore, do you know some books or links to suggest?

Improve this question
$\endgroup$
1
  • $\begingroup$ Please provide a graph (real units do not matter). $\endgroup$
    – AlainD
    Aug 31, 2018 at 8:51

3 Answers 3

Reset to default
1
$\begingroup$

For understanding the seasonality of time series data I would start with Holt-Winters Method or ARIMA. Understanding these algorithms will help with understand how time series forecasting works. Time series forecasting

For unsupervised classification, I would start with something like k-means clustering for anomaly detection.
Anomaly Detection with K-Means Clustering

These links should be a good starting point, I hope this helps.

Improve this answer
$\endgroup$
3
  • 4
    $\begingroup$ As a general notice, there has been comprehensive research that prooves clustering subsequences in time-series is meaningless. The work that is linked here also has a reference to this piece of research. So I believe this approach could not be a good starting point as it is proved to be absurd. I found algorithms in Luminol's library much suited as a starting point for anomaly detection. $\endgroup$
    – Novin Shahroudi
    Jul 17, 2019 at 11:17
  • $\begingroup$ @NovinShahroudi, could you go into detail as on why this is meaningless, or provide a link? Can't find it in OP's question... $\endgroup$
    – Thomas
    Aug 22, 2019 at 7:52
  • 3
    $\begingroup$ @Thomas you can find it here: cs.ucr.edu/~eamonn/meaningless.pdf $\endgroup$
    – Novin Shahroudi
    Aug 22, 2019 at 10:22
5
$\begingroup$

You can have a look here, where many open-source algorithms specifically for anomaly detection on time-series data (e.g. metrics) are collected, both for online of offline settings.

Almost all of them are unsupervised approaches that require no labels to detect the anomalies.

They also automatically handle some of the issues you mentioned, like seasonality.

Improve this answer
$\endgroup$
0
$\begingroup$

About anomalies detection, you have a bunch of methods. In the jargon they are called outliers, and Wikipedia's Outlier article is a very good start.

Most answers from Time Series will advise to use an Exponential smoothing (in the Holt-Winters version to take care of the seasonality), or the *ARIMA (of which Exponential smoothing is a individual case).

This may be good. However, for situation like this, Engineers are used to filter the sensor with a Kalman filter. The mathematical step is steeper, though.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.