The general formula for absolute discounting for calculating language model probabilities subtracts a discounting parameter d from the count of the ngram before calculating the probabilities.

LM with absolute discounting

The perplexity of the model first decreases as d increases, and then later starts increasing again.

Perplexity vs Discounting Parameter

What is the reason behind this change? Why does the perplexity decrease first and then start increasing again?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.