3
$\begingroup$

Let's say we have feature A, B, C, D, E to represent one observation in an outlier detection model. We are using scikit-learn outlier detection in our case.

AFAIK, if we normalize all the features, they are as equally important as others.

What if I want to make B very important, e.g. if A, C, D, E increase 20%, the observation might still be considered as normal case while B increases 10%, the observation needs to be marked as an outlier.

Is there any way to increase or decrease features' weight in outlier detection model?

$\endgroup$
3
  • $\begingroup$ After normalization multiply B by 2, or at least the difference between B and the expected value for B $\endgroup$
    – Pieter21
    Sep 12 '17 at 12:41
  • $\begingroup$ how do I know 2 is the best number to multiply? $\endgroup$
    – Shengjie
    Sep 13 '17 at 1:07
  • $\begingroup$ 2 is the factor between the 10% and 20% you mentioned. I'll answer with a more generic method. $\endgroup$
    – Pieter21
    Sep 13 '17 at 7:51
1
$\begingroup$

Please look at my answer at Identfying spikes in data .

A general method could be to calculate a (moving) average and standard deviation. Outliers are points that differ more than 3 times the standard deviation.

In your case, B will probably have a standard deviation of 3%, because 10% is considered an outlier. For A, C, D, E, the standard deviation may be 7%, because 20% is not considered an outlier.

Relative importance does not make something an outlier. A large deviation does.

$\endgroup$
2
  • $\begingroup$ I am using outlier detection directly from scikit-learn. So my question is that in order to increase B's importance, what I can do before I feed all the features to the algorithm? $\endgroup$
    – Shengjie
    Sep 17 '17 at 13:38
  • $\begingroup$ @Shengjie What would 'importance' mean in outliers? Outliers mean that they deviate from the expected values. $\endgroup$
    – Pieter21
    Sep 17 '17 at 19:46
0
$\begingroup$

The answer depends on what exactly outlier detector you use.

EllipticEnvelope and IsolationForest authomatically adjust to the scale and correlations between variables, so you cannot easily change relative weights of features. With IsolationForest you can increase importance of B by just replicating this column several times and so icreasing its relative frequency in the tree splits.

Other algorithms, such as LocalOutlierFactor and kernel-based OneClassSVM are explicitly based on the Euclidean distance between the points. And distance depends on feature scaling. It means that you can multiply B by some factor after normalization, and its importance will increase by this factor.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.