I need help, are there any practical implications of a dominant principal component. For example, if of three PCs, PC1 explains almost 100% of the variance in this dataset, What does this mean in practical terms? or what does it say about the dataset?

Any help is highly appreciated. Thanks!

  • $\begingroup$ Please provide more explanation like what is three PCs(does it mean three features?),PC1, possibly after that I can provide some answer. $\endgroup$ – vipin bansal Jun 27 '19 at 9:57

The principal components describe the amount of the total variance that can be explained by a single dimension of the data.

This is equivalent to the spread of the datapoints in a given dimension. The dimensions are (of course) direction that are orthogonal i.e. at 90 degrees to one another.

Have a look at this example of data points, where the red lines show the breasth of the data in two dimensions:

simple example of PCA

The dimensions don't have to be in X and Y - they could be pointing in any direction, but must be orthogonal. (more detail in this answer)

We can clearly see a bigger spread in the horizontal X-dimension, so I might expect it to account for 80% of the variance in the dataset. The vertical Y-dimension has less variance, less spread, so explains a smaller amount of the total variance. In this simple 2d example, it would explain the remaining 20% of the variance (it must sum to 100%).

In practical terms, if principal components have all very similar values, you might expect the data to form a circle (in 2d), and this means there is little directionality in the feature-space. You might like to think in terms of correlation between the features; a movement in one direction of the space does not guarantee a movement of a certain direction in the second feature. The opposite would be true if e.g. the first component had a normalised value of ~1 i.e. explained approximately 100% of the variance.

I say normalised, because the raw values that come out of PCA do not necessarily between 0 and 1 - so you can normalise them to help interpretation.

In a higher dimensional space, say with 10 variables (so 10d feature space), PCA computes eigenvectors and eigenvalues, which look for orthogonal dimensions that explain the variance of the data points, but these are not all all constricted to the dimensions of your features themselves! This means that you cannot just say that the first component (e.g. with a value of 0.6) is there because of a feature X, i.e. not due to a single feature, but a mixture of the features.

About the dataset

If the first PC explains nearly all of the data's variance, it is likely that you can express you data a lot more succinctly. PCA is often used as a dimensionality reduction method - so in this extreme case, you can reduce a high-dimensional space and convert it to a lower dimensional space without (theoretically) losing much of the explanatory power i.e. a model should be able to learn as much about the feature space from the one predominant PC, compared to the rich high-dimensional space. This of course saves compute time for any model.

I said *theoretically) above, because using PCs to model will make the interpretation of the resulting model itself - in terms of the original input features - more difficult.

For a more thorough introduction, read through a blog post like this one. For some practical examples, check out the Sci-kit learn documentation.

| improve this answer | |
  • $\begingroup$ hi, thanks for the elaborate explanation. I know it is the case that PC1 is in the direction of maximum variance (standard 70-80%) but are there any practical implications if PC1 explains 95 - 100% of the variance in the dataset? does it suggest anything about the dataset in question? I may be mistaken in thinking there is something up here, but just need a second opinion. $\endgroup$ – Chidi Jun 27 '19 at 10:23
  • $\begingroup$ @Notable1 - please see my addition. $\endgroup$ – n1k31t4 Jun 27 '19 at 11:52
  • $\begingroup$ @Notable1 - if my response answered your questions, please consider marking it as answered with the green tick :) $\endgroup$ – n1k31t4 Jun 29 '19 at 17:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.