1
$\begingroup$

While studying Gaussian mixture models and the expectation maximization algorithm, I also came across a number of studies that used 'Discriminative training' for the models that addressed some limitations of the EM algorithm. In particular, the use of maximum mutual information is quite common.

I understand the basic idea behind maximizing mutual information; however, I am unable to find any source that gives the equations involved in estimating the means, variances, mixing parameters and the predicted labels.

Can you please state them along with an explanation?

$\endgroup$
2
  • $\begingroup$ MLE is the usual thing. The one paper I found that "introduced" MMI has 4 citations. Why do you think this is "quite common"? $\endgroup$ Jun 11 '18 at 6:13
  • $\begingroup$ I've seen more than one publication using mmi. It is quite common in speech recognition. $\endgroup$ Jun 11 '18 at 6:58
2
$\begingroup$

Multiclass Discriminative Training of i-vector Language Recognition, this article contains update formulas for mean and covariance (diagonal).

First you should obtain parameter estimations using ML (so calculate mean and covariance from class data) and then iteratively update them according to formulas (16-20). I'm not sure about the meaning of C0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.