What are the cases when we should not use PCA for dimensionality reduction and what to use in such cases?
You should not use PCA if you only have categorical variables, and thus the distance function in PCA is invalid.
Correspondence analysis is a common alternative.
PCA is a linear transformation of your variables to a set of uncorrelated ones.
It is good when you want to remove redundancy (in a linear sense) in your data but it is bad if you want to uncover the "true cause" of the variables. Furthermore the causes can be non-linear in nature and the behavior of your data cannot be captured using a linear model.
Then you would stay away from PCA and consider e.g. non linear dimensionality reduction methods