1
$\begingroup$

While studying Gaussian mixture models and the expectation maximization algorithm, I also came across a number of studies that used 'Discriminative training' for the models that addressed some limitations of the EM algorithm. In particular, the use of maximum mutual information is quite common.

I understand the basic idea behind maximizing mutual information; however, I am unable to find any source that gives the equations involved in estimating the means, variances, mixing parameters and the predicted labels.

Can you please state them along with an explanation?

Improve this question
$\endgroup$
2
  • $\begingroup$ MLE is the usual thing. The one paper I found that "introduced" MMI has 4 citations. Why do you think this is "quite common"? $\endgroup$
    – Has QUIT--Anony-Mousse
    Commented Jun 11, 2018 at 6:13
  • $\begingroup$ I've seen more than one publication using mmi. It is quite common in speech recognition. $\endgroup$
    – Sairaam Venkatraman
    Commented Jun 11, 2018 at 6:58

1 Answer 1

Reset to default
2
$\begingroup$

Multiclass Discriminative Training of i-vector Language Recognition, this article contains update formulas for mean and covariance (diagonal).

First you should obtain parameter estimations using ML (so calculate mean and covariance from class data) and then iteratively update them according to formulas (16-20). I'm not sure about the meaning of C0.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.