3
$\begingroup$

I was investigating scikit-learn's implementation of the EM algorithm for fitting Gaussian Mixture Models and I was wondering how they did come up with using the average log likelihood instead of the sum of the log likelihoods to test convergence.

I see that it should cause the algorithm to converge faster (given their default parameters), but where does that idea come from ?

Does anyone know if they based this part of the implementation on a specific paper or if they just came up with it and used it ?

In most explanations of the EM algorithm I have come across, they would have used log_likelihoods.sum() instead of log_likelihoods.mean().

Improve this question
$\endgroup$
1
  • 2
    $\begingroup$ Welcome to DataScience.SE! $\endgroup$
    – Emre
    Jul 1, 2016 at 22:01

1 Answer 1

Reset to default
4
$\begingroup$

It makes unit testing easier; invariant to the size of the sample.

Reference: the github discussion that led to the change.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @Dawny33 no hard feeling, but I still don't understand why you rejected the edit. I just believe that what I'm saying states the same thing as Emre but with clearer. :) $\endgroup$
    – eliasah
    Jul 3, 2016 at 12:33
  • $\begingroup$ @Emre what do you think ? I think your answer is great but it just need some clarifications. $\endgroup$
    – eliasah
    Jul 3, 2016 at 12:34
  • $\begingroup$ @eliasah many of your edits are putting lots of words in the answerer's mouth. They may or may not be right; they shouldn't be an edit. Write your own answer, or comment. $\endgroup$
    – Sean Owen
    Jul 3, 2016 at 13:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.