4
$\begingroup$

I am having some difficulty in seeing connection between PCA on second order moment matrix in estimating parameters of Gaussian Mixture Models. Can anyone connect the above??

$\endgroup$
2
  • $\begingroup$ Please provide more context here. Consider providing an example or the task that you are interested in. Why is it that you think there is a connection in the first place? May be cite a source? $\endgroup$
    – Nitesh
    Nov 25, 2014 at 3:53
  • $\begingroup$ The task is to find clusters (non parametric) for given set of data points. So I am interested to find mean components in the case of GMM. The claim is these mean components lie in the span of eigen vectors of second order moment matrix . Can you give me some intuition behind this claim. $\endgroup$
    – tejaswi
    Nov 25, 2014 at 5:11

1 Answer 1

5
$\begingroup$

I believe the claim that you are referring to is that the maximum-likelihood estimate of the component means in a GMM must lie in the span of the eigenvectors of the second moment matrix. This follows from two steps:

  1. Each component mean in the maximum-likelihood estimate is a linear combination of the data points. (You can show this by setting the gradient of the log-likelihood function to zero.)
  2. Any linear combination of the data points must lie in the span of the eigenvectors of the second moment matrix. (You can show this by first showing that any individual data point must lie in the span, and therefore any linear combination must also be in the span.)
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.