I know that PCA is good in differentiating between anomalies and normal data and it helps to differentiate between them when it tries to transfer the data to another dimension. I mean it can somehow maximize the separation between the regular and irregular data points. I also have seen this in my codes. However, someone has asked me to prove it! by a simple detailed example (e.g, a simple 2*2 matrix)? Can anyone help?

Thanks in advance.

  • $\begingroup$ Very interesting question, I don't know if I can come up with a solution but I'll definitely give it a thought and ask around. My intuition tells me that it has to do with proving how eigen values define the space and that dives deep into linear algebra which is not "simple". $\endgroup$ Apr 25, 2018 at 5:31
  • $\begingroup$ This should answer your question : math.union.edu/~jaureguj/PCA.pdf $\endgroup$
    – moksha
    Apr 25, 2018 at 7:35
  • $\begingroup$ Thanks, Haris. Please let me know if you found sth $\endgroup$
    – Arkan
    Apr 25, 2018 at 7:43
  • $\begingroup$ Thanks, moksha. In the meantime, do you have any point to mention as the main reason? $\endgroup$
    – Arkan
    Apr 25, 2018 at 7:44

1 Answer 1


The first eigenvector of the PCA have the direction of the maximum variance of the data. Then the second follows with the direction of the next highest variance and so it continues....

The easiest way to think about this, by my experience, is with 2D data and transform it to 1D. Visualization can be found here: PCA Visualization

It does not maximize the separation between the regular and irregular data points. It does dimension reduction, by trying to explain as much of the variance as possible by creating a new basis, eigenvectors(directions). However, it is not very robust to outliers. Imagine all your datapoints are basically on a line, now suddenly you receive an abnormal value. As this abnormal value will have an impact on the variance, the eigenvalue directions will change, and with that the values of the eigenvectors!

For anomaly detection: You can use the PCA eigenvectors of "normal data" against thos of new data. Simply do distance calculations between the eigenvectors. If you have a big distance => Anomaly

There are methods such as robust PCA that better cope with noisy data and can find the outliers.

Hope this helps.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.