PCA removes the connection with the original features,so the interpretation of the visualisations in the principle component space is therefore not very meaningful.
E.g. cluster A has higher values of PC1, where cluster B has higher values of PC2.
If you can clearly see that PC1 is only representative of Feature X, then fine, but this isn't often the case.
In my opinion, the following information can be considered as implicit assumptions of PCA:
Data comes from a constant multivariate normal distribution.
Principal components are orthogonal.
More variances have an important structure.
You seem to be using a generic eignvalue solver. Perhaps your matrix is badly conditioned and the aglorithm cannot pick it up. Lots of zero eigenvalues with few massive ones certainly seem suspicious. Have you tried solvers specific for hermitian/symmetric matricies? eigh & eigvalsh
Also, I would z-score all your features, and drop all features with zero ...