I've read some posts about PCA applied on time series, but still a bit confused and I have the following questions(Suppose I am working with a time series of the return of 50 industries and I want to use a clustering algorithm to divide them into several group):

  1. Say I have calculated the eigenvalue and eigenvector from the correlation matrix, and found that the first twenty eigenvalue account for 85% of the total, and I then use these twenty eigenvalue to approximated the original time series. I know if I choose all eigenvalues then I can get the identical original time series, but what information did I lose if I choose twenty of them specifically? What is the purpose to do so?

  2. I found some post said we can always drop the first principle component(means we don't use it), why can we do that?

  3. Can I interpret each eigenvalue as a trend in the market, for example the first principle component, can I derive whether the corresponding industry is in the same direction or different from the market trend based on the sign of its corresponding eigenvector, and if so, can I apply k-means to all industries by using the eigenvectors of the first few principle component to group them, is this make sense?

Welcome for any kind of hint or ideas, thanks.


1 Answer 1


Usually, the norm to asking any question on Stack Overflow or any other sister websites is that one is supposed to ask one question only unless they are very similar which doesn't seem to be in this case.

To answer your questions, $X^TX$ is called Sample Covariance (or Correlation) Matrix where $X$ is the data matrix of dimensionality $(m,d)$. So, the resultant matrix has a dimensionality of $(d,d)$ where $d$ is the dimensionality of the feature space.

And as you said, this matrix is made to go an eigendecomposition to get $w\cap w^{-1}$ where $\cap$ is a diagonal matrix of eigenvalues, arranged in decreasing order and $w$ is the normalized eigenvectors stacked according to the corresponding eigenvalues. The reason one might want to choose some $k$ number of dimensions is to reduce dimensionality.

Reduced dimensionality provides multiple benefits - reducing space complexity, faster computations etc. The problem arises when you specifically talk about time series. PCA, ICA does not take into account temporal dependence which might cause data to have suboptimal forecast. There are different ways to tackle this issue, one might want to use Forecastable Component Analysis, autoencoders etc to ensure they are reaping the benefits of not only gaining the benefits that PCA provides but also ensuring that the problems with PCA's are avoided.

To answer the second question, I'm NOT sure of the reason but one may want to drop the first principal because it is in the direction of maximum variance, i.e. it varies most in this direction.

  • $\begingroup$ I should delete that question before post it on Stack Exchange, sorry about that, it's just because been three days since it posted and still no one has responded. $\endgroup$
    – Carl
    Mar 19, 2021 at 13:12
  • $\begingroup$ Don't worry about deleting it, if it really is against the rules the mods will delete it, just keep it in mind whenever you're posting it, it's all fine. You might also want to ask the same question on CrossValidated, maybe you'll find some more responses there. $\endgroup$ Mar 19, 2021 at 13:14

Not the answer you're looking for? Browse other questions tagged or ask your own question.