4
$\begingroup$

A few work colleagues and I were looking through a recently replaced 'cat', we had in the workplace. For those of you which are curious, the 'cat' in this context, refers to a specific type of pump known as a cat pump. It's affectionally called the 'kitty' as we only have one of these pumps.

After one routine inspection, I noticed a number of periods where kitty wasn't performing well. This is labelled in the dataset as Cat Anomaly.

(Accessible from here if anyone wishes to analyse further:

https://drive.google.com/open?id=1e0OhzhaSZP9_A3QdaK9-BvWXbRPZacED)

This is what it looks like when it is abnormal:

Essentially, it looks similar to a sinusoidal pattern and is slightly curved. I also noted that the cluster of points seem to be quite tight when it is an abnormal state.

Abnormal Cat Behaviour Circled

Using Python's Sklearn Libraries and some good ol fashioned signal processing, I decided to remove some of the noisy clusters so I can see what the signal looks like 'without' the noise. To do this, I have performed a Savitzy-Golay Filter on the data, which has smoothed this out as per below.

Cleaned Signal using SG Filter

I have then used a feature to subtract the values from the 'normal' cat values - 'smoothed' cat values which gives me a rough indication of when the 'clusters' appear.

This has worked acceptably (~90% accuracy) and I can pick up abnormalities to an extent when I feed this feature into my random forest model, but I feel the 'cluster' approach I have used is very clunky and there are much better ways to capture the 'noisiness' of the cat signal as a variable.

ROC Curve indicating Model Accuracy

So now I'm looking for a better way of recognizing and separating 'abnormal' cat behaviour.

So, as a better feature or variable to capture 'noisy' clusters, does anyone else have any ideas on improvements on how to capture this?

Improve this question
$\endgroup$
3
  • $\begingroup$ Anyone open to the challenge? $\endgroup$
    – IronKirby
    Sep 26, 2018 at 0:13
  • $\begingroup$ Out of curiosity, why are you using a random forest? It appears that you have time series data, so I am wondering why you are not using a time series model? Have you considered a Recurrent Neural Network? $\endgroup$
    – Skiddles
    Oct 24, 2018 at 22:16
  • $\begingroup$ Interesting suggestions! The problem I'm looking at is a classification issue. Random Forests are very well suited towards this due to the rapid speed of operations and low computation costs. Why would a recurrent Neural Network be of assistance in this context? I've noted RNNs are used significantly in image classification due to the large volume of data required. $\endgroup$
    – IronKirby
    Oct 25, 2018 at 5:25

1 Answer 1

Reset to default
1
$\begingroup$

At what resolution are you performing you classification? At sample level?

If so, what does your baseline model look like? Did you try for example just feeding the last 5-10 raw samples as features and see what accuracy you get that way?

Also, you could try seeing what happens if you do some kind of "smoothing" at the output - e.g. if you assume that one "good" sample, surrounded by 10 "bad" samples should be reclassified as "bad" also -- because the system cannot jump so quickly between normal and faulty modes of operation.

Finally (or should be firstly), what is the accuracy that is actually necessary for your model to be useful in practice?

Improve this answer
$\endgroup$
2
  • $\begingroup$ Hi, So firstly, the model accuracy at the moment is already acceptable (>90%), and we are picking events up. However, I am looking for alternative features to capture the 'fuzziness' of the clusters.Secondly, what do you mean when you say, 'feed the last 5-10 raw samples as features', are you referring to using lagged variables? $\endgroup$
    – IronKirby
    Sep 24, 2018 at 23:05
  • $\begingroup$ Indeed, lagged variables. :-) $\endgroup$
    – Nemanja Radojković
    Oct 26, 2018 at 7:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.