Language models are used extensively in Natural Language Processing (NLP) and are probability distributions over a sequence of words or terms.
Language models are used extensively in Natural Language Processing (NLP) and are probability distributions over a sequence of words or terms. Commonly, language models are constructed to determine the probability of any given word given the set of n previous words. A popular language model is an n-gram one which has two variations: unigram and bigram.
The unigram model (Bag of Words, n=1):
$P_{unigram}(w_1,w_2,w_3,w_4) = P(w_1)P(w_2)P(w_3)P(w_4)$
The bigram model (n=2):
$P_{bigram}(w_1,w_2,w_3,w_4) = P(w_1)P(w_2|w_1)P(w_3|w_2)P(w_4|w_3)$
Other more sophisticated methods for constructing language models also exist using Exponential and Neural Networks.
