Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.
Sign up to join this community
I know that PCA is good in differentiating between anomalies and normal data and it helps to differentiate between them when it tries to transfer the data to another dimension. I mean it can somehow maximize the separation between the regular and irregular data points. I also have seen this in my codes. However, someone has asked me to prove it! by a simple detailed example (e.g, a simple 2*2 matrix)? Can anyone help?
Thanks in advance.
The first eigenvector of the PCA have the direction of the maximum variance of the data. Then the second follows with the direction of the next highest variance and so it continues....
The easiest way to think about this, by my experience, is with 2D data and transform it to 1D. Visualization can be found here: PCA Visualization
It does not maximize the separation between the regular and irregular data points. It does dimension reduction, by trying to explain as much of the variance as possible by creating a new basis, eigenvectors(directions). However, it is not very robust to outliers. Imagine all your datapoints are basically on a line, now suddenly you receive an abnormal value. As this abnormal value will have an impact on the variance, the eigenvalue directions will change, and with that the values of the eigenvectors!
For anomaly detection: You can use the PCA eigenvectors of "normal data" against thos of new data. Simply do distance calculations between the eigenvectors. If you have a big distance => Anomaly
There are methods such as robust PCA that better cope with noisy data and can find the outliers.
Hope this helps.
