I am having some difficulty in seeing connection between PCA on second order moment matrix in estimating parameters of Gaussian Mixture Models. Can anyone connect the above??

  • $\begingroup$ Please provide more context here. Consider providing an example or the task that you are interested in. Why is it that you think there is a connection in the first place? May be cite a source? $\endgroup$ – Nitesh Nov 25 '14 at 3:53
  • $\begingroup$ The task is to find clusters (non parametric) for given set of data points. So I am interested to find mean components in the case of GMM. The claim is these mean components lie in the span of eigen vectors of second order moment matrix . Can you give me some intuition behind this claim. $\endgroup$ – tejaswi Nov 25 '14 at 5:11

I believe the claim that you are referring to is that the maximum-likelihood estimate of the component means in a GMM must lie in the span of the eigenvectors of the second moment matrix. This follows from two steps:

  1. Each component mean in the maximum-likelihood estimate is a linear combination of the data points. (You can show this by setting the gradient of the log-likelihood function to zero.)
  2. Any linear combination of the data points must lie in the span of the eigenvectors of the second moment matrix. (You can show this by first showing that any individual data point must lie in the span, and therefore any linear combination must also be in the span.)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.