0
$\begingroup$

I have relatively little knowledge of unsupervised machine learning.

I'm working on a project that aims to find anomalies in a set of n data, measured every x ms, and we have no label on all these data. So I'm here to ask you which algorithms/approach should be the best?

Edit : I'm working in an industrial context, and my data are sensor readings (temperature, pressure, power consumption, etc.) all from the same equipment.

Improve this question
$\endgroup$
  • $\begingroup$ There are very different kinds of time series (e.g., audio vs. cardiogram vs. website visitor count) that all can have very different kinds of anomalies (e.g., server down or DoS flood on web sites) and that may be trivial to detect, or require very complicated algorithms. You need to provide much more detail, and make some assumptions about what normal behavior is. There is no "one size fits all" approach. $\endgroup$ – Has QUIT--Anony-Mousse Mar 4 '19 at 19:23
1
$\begingroup$

One option is to treat it as a "supervised learning" problem.

Since

a set of n data, measured every x ms

You have a time series. This can be treated as a supervised problem.

Say that you have observations for 10 minutes. In supervised learning, each sample would be :

  • X : Set of last t-1 , t-2, t-3 and t-N observations (N is fixed, say 300)
  • Y : Observation t

You can train a model to predict next observation, given last 300 observations. If actual value is beyond a certain % from the prediction, it is an anomaly.

Example : https://machinelearningmastery.com/multivariate-time-series-forecasting-lstms-keras/

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you for your answer, I'm going to study this solution ! $\endgroup$ – M. Allspach Mar 4 '19 at 8:20
0
$\begingroup$

If you DO NOT have hundreds of features, then consider k-Nearest Neighbour (kNN) based outlier detection.

For a window of w unit times

  1. calculate the median distances for the features (power,temp,..etc)
  2. For each time stamp, calculate the kNN distance

Then you can select top 1% data points with highest deviations in kNN distances compared to the median kNN distance.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.