# About

Tag Info

Language models are used extensively in Natural Language Processing (NLP) and are probability distributions over a sequence of words or terms.

Language models are used extensively in Natural Language Processing (NLP) and are probability distributions over a sequence of words or terms. Commonly, language models are constructed to determine the probability of any given word given the set of n previous words. A popular language model is an n-gram one which has two variations: unigram and bigram.

The unigram model (Bag of Words, n=1):

$$P_{unigram}(w_1,w_2,w_3,w_4) = P(w_1)P(w_2)P(w_3)P(w_4)$$

The bigram model (n=2):

$$P_{bigram}(w_1,w_2,w_3,w_4) = P(w_1)P(w_2|w_1)P(w_3|w_2)P(w_4|w_3)$$

Other more sophisticated methods for constructing language models also exist using Exponential and Neural Networks.