1
$\begingroup$

The general formula for absolute discounting for calculating language model probabilities subtracts a discounting parameter d from the count of the ngram before calculating the probabilities.

LM with absolute discounting

The perplexity of the model first decreases as d increases, and then later starts increasing again.

Perplexity vs Discounting Parameter

What is the reason behind this change? Why does the perplexity decrease first and then start increasing again?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.