3
$\begingroup$

I have a problem for which I have not been able to find any answers in my search so far.

BACKGROUND

I am working on an anomaly detection problem on machines utilising an auto-encoder. I am building a model file per machine because the machines' temporal behaviour varies quite a lot.

I have 5 features:

  • Numerical integer ranging between 0 and x (x varies per machine)
  • The other 4 features are categorical (After trying LabelEncoding, my architecture prefers One-Hot encoding)

I have tried to scale the numerical feature using Normalisation (MinMaxScaler and StandardScaler) which did not yield very good results at all.

As an alternate to scaling the inputs - I decided to scale the outputs using MinMaxScaler from scikit. This is so that I can have 1 generic threshold I can apply across the different models to identify anomalies.

Although this has yielded the best results so far - in practice, the outputs become too polarised to either the 0 or the 1 and consequently I am missing outliers that I shouldn't be.

PROBLEM

What scaling technique(s) can I use on the output from my auto-encoder such that I can apply 1 generic threshold across all models to identify anomalies?

$\endgroup$
  • $\begingroup$ What's your output? Is it the distance between the original example, and its reconstruction after passing it through the auto-encoder? I'm asking to know the potential range of the output. $\endgroup$ – Paul Jun 5 at 12:34
  • $\begingroup$ @Paul Yes, exactly. $\endgroup$ – Ash Jun 10 at 13:07
0
$\begingroup$

I always recommend to use StandardScaler() rather than MinMaxScaler(). The reason is that the latter is too sensitive to outliers. Imagine shrinking the given vector in the [0, 1] interval:

[1, 2, 3, 4, 999]

Most of your data would be squeezed in a tiny subspace of the [0, 1] interval, causing absurd compression. The problem doesn't exist if you use the StandardScaler(). This looks particularly relevant given that you're trying to detect abnormal observations. (I would use min-max scaling only if I'm sure that a given variable is bounded within a given range of values.)

But most importantly: What are the problems of your output? As far as we know, they might not be due to scaling.

$\endgroup$
  • $\begingroup$ thanks! I have used StandardScaler and precisely due to the reason that you mentioned above - it is less sensitive to outliers - is why it hasn't performed how well the MinMaxScaler does. My original problem is not scaling - it is the fact that I need a new decision boundary for what is anomalous and what is normal for every machine. If I can scale the reconstruction output from every machine - I can use 1 generic threshold value. However, I am currently trying to find a unique threshold for every machine instead without scaling. $\endgroup$ – Ash Jun 10 at 13:12
0
$\begingroup$

What scaling technique(s) can I use on the output from my auto-encoder such that I can apply 1 generic threshold across all models to identify anomalies?

Bad idea.

Because:

I am building a model file per machine because the machines' temporal behaviour varies quite a lot.

An Auto-Encoder will try to learn the general representation of what a machine output (or its performing stats - or whatever you can get as numbers from the machine behaviour) and understand that as the general image of the machine. This is temporal so if you have two machines that:

  • machine A: does almost nothing all morning, executes at almost 100% CPU everyday around 1PM, works at 10-20% CPU during the afternoon.
  • machine B: works at 50-60% all day long.

This is a simplistic problem but:

  • CPU at 50% at 9AM is an anomaly to machine 1, but definitely not an anomaly for machine 2.
  • CPU at 3% is always an anomaly for machine B, but definitely not an anomaly for machine A in the morning.

No scaling will make these two anomalies fit together on a single scale (one can perform a thought experiment of graphing both daily CPU usages and then try to perform functional transforms to both functions in order to make them look the same).

The fact that one has very different values from the other at different times make the relation non-linear. Moreover the relation cannot be equated in trivial non-linear fashion either, for example: to equate the derivatives of both CPU usage functions one would need derivatives so high that both derivatives would be zero - meaning that one would lose all information about the data at that point.

$\endgroup$
  • $\begingroup$ thanks! I have essentially come to the same conclusion - but a very good example. I am currently trying to identify the reconstruction threshold that distinguishes an anomaly for every model instead of trying to scale the results. $\endgroup$ – Ash Jun 10 at 13:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.