In language modeling, L(lambda1, lambda2, lambda3) is defined as:

Sum(count of trigram(u,v,w) x q(w|u,v)) 

where u, v, w are words in the corpus and perplexity is defined as:



l = (1/M)Sum(log(q(w|u,v)).

where M is the total no. of words in the corpus.


q(w|u,v) = lambda1*q(w|u,v) + lambda2*q(w|v) + lamdba3*q(w)

The q-values on the right hand side are maximum likelihood estimates. Some of the materials for Natural Language Processing state that maximizing L is same as minimizing perplexity. I don't see how that is true or can be mathematically proved.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.