I know that PCA is good in differentiating between anomalies and normal data and it helps to differentiate between them when it tries to transfer the data to another dimension. I mean it can somehow maximize the separation between the regular and irregular data points. I also have seen this in my codes. However, someone has asked me to prove it! by a simple detailed example (e.g, a simple 2*2 matrix)? Can anyone help?

Thanks in advance.

  • $\begingroup$ Very interesting question, I don't know if I can come up with a solution but I'll definitely give it a thought and ask around. My intuition tells me that it has to do with proving how eigen values define the space and that dives deep into linear algebra which is not "simple". $\endgroup$ – Haris Nadeem Apr 25 '18 at 5:31
  • $\begingroup$ This should answer your question : math.union.edu/~jaureguj/PCA.pdf $\endgroup$ – moksha Apr 25 '18 at 7:35
  • $\begingroup$ Thanks, Haris. Please let me know if you found sth $\endgroup$ – Arkan Apr 25 '18 at 7:43
  • $\begingroup$ Thanks, moksha. In the meantime, do you have any point to mention as the main reason? $\endgroup$ – Arkan Apr 25 '18 at 7:44

The first eigenvector of the PCA have the direction of the maximum variance of the data. Then the second follows with the direction of the next highest variance and so it continues....

The easiest way to think about this, by my experience, is with 2D data and transform it to 1D. Visualization can be found here: PCA Visualization

It does not maximize the separation between the regular and irregular data points. It does dimension reduction, by trying to explain as much of the variance as possible by creating a new basis, eigenvectors(directions). However, it is not very robust to outliers. Imagine all your datapoints are basically on a line, now suddenly you receive an abnormal value. As this abnormal value will have an impact on the variance, the eigenvalue directions will change, and with that the values of the eigenvectors!

For anomaly detection: You can use the PCA eigenvectors of "normal data" against thos of new data. Simply do distance calculations between the eigenvectors. If you have a big distance => Anomaly

There are methods such as robust PCA that better cope with noisy data and can find the outliers.

Hope this helps.


Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.