What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
2 Answers
You should not use PCA if you only have categorical variables, and thus the distance function in PCA is invalid.
Correspondence analysis is a common alternative.
PCA is a linear transformation of your variables to a set of uncorrelated ones.
It is good when you want to remove redundancy (in a linear sense) in your data but it is bad if you want to uncover the "true cause" of the variables. Furthermore the causes can be non-linear in nature and the behavior of your data cannot be captured using a linear model.
Then you would stay away from PCA and consider e.g. non linear dimensionality reduction methods