1
$\begingroup$

I have many (around 40) separate time-series from different sensors, each measuring magnetic field intensity. I am looking to get an ML algorithm to identify a particular anomaly. This anomaly happens infrequently (a handful of times in 25 years of 60-second data, depending on the sensor) and is only apparent when you look at two time-series side-by-side:

figure_zzz

The yellow circle is the anomaly, and can be described roughly like this: when the time series are particularly disturbed, the red time-series will briefly anti-correlate with the green time-series over a period of a few hours (data are sampled every 60s). This can easily be done by hand for a pair of time-series.

The Plan

I'd like to train a machine learning algorithm by manually identifying these anomalies for a sensor pair, and then set the trained algorithm on the rest of the pairs of time-series

The Problem

I have no idea what the best algorithm would be for this problem, as it involves two time-series. I messed around with gathering information in 6 hour blocks and treating it like a non-time series problem. This involved treating each 6-hour block separately (i.e., each with std, average, max, min of each time-series as well as a correlation coeff and a 1 or 0 output determined manually), but I feel like there is a neural network that will be able to better figure out the important variables.

The Question

What would be the best approach to this problem in order to train a ML algo? Is my approach above suitable?

Improve this question
$\endgroup$
2
  • $\begingroup$ Is your data labelled? Like, anomaly vs not anomaly? $\endgroup$ – David Masip Jul 17 '18 at 8:03
  • $\begingroup$ Did you ever find a ML algorithm to deal wit this issue? In your diagram, and from your description, it would seem that the area around the last large spike in the red data would be anomalous too - is it, or not? $\endgroup$ – Ian Apr 6 '20 at 1:04
1
$\begingroup$

Before think which algorithm, think what features. You may add/ignore features that imply your problem fluently.

This will be around quantifying anomalies (new variable?). Try to add a residual variable? Or simply feature different between red and green variables?

Improve this answer
$\endgroup$
1
$\begingroup$

If your goal is just to know if two time series has this type of anomaly, then this is a classification problem. It is good to try to manually label pairs of series, and then I will use LSTM algorithm for the problem with time-steps from the two series. Up to you to experiment to know how long the time block will be.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.