1
$\begingroup$

When using the EM algorithm in Gaussian Mixture Models (GMM), in the E-step, we take each x set in the training dataset to calculate and update the "weight" and parameters of each Gaussian distribution of the clusters (M-step). I have read that we do this until it converges. I am a little confused here. Does that mean it loops through the whole training dataset X every time in "one step" of the EM algorithm? Or is "one step" corresponding to calculating ONE particular x set in the whole dataset and using it to update the parameters and weights of each distribution?

To make the question more clear, does each "step" in the EM algorithm in GMMs involve the whole training dataset X or does each step mean one particular x set in the training set, and we loop over the dataset one at a time to improve one at a time?

Any help will be appreciated! Thanks.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

At the beginning of the algorithm, you are taking an initial guess for the relevant parameters. Then in each iteration, in the E-step you are computing the responsibilities for all the given datapoints and in the M-step you are computing the weighted means and variances.

Since you are clustering the datapoints there is not training or test set. You just want to divide your dataset to two (or more) clusters.

Improve this answer
$\endgroup$
2
  • $\begingroup$ I see. Just to make sure, so technically after initializing some parameters, we will go through the whole dataset X in every iteration to update and calculate the responsibilities and parameters. We stop when it converges, meaning it has reached the optimized parameters that describe the dataset. Correct? $\endgroup$
    – YCCCCC
    Commented Sep 17, 2019 at 21:56
  • 1
    $\begingroup$ Yes. If you cannot meet the stopping criterion, you may define a maximum number of iterations. $\endgroup$
    – Christos Karatsalos
    Commented Sep 18, 2019 at 22:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.