mathematical accurate definition of the binary independence model

Question

I have a hard time understanding the exact mathematical meaning behind the binary independence model. On wikipedia we can see the following definition or similarly in the book from Manning and Schütze,

it claims that

The probability P(R|d,q) that a document is relevant derives from the probability of relevance of the terms vector of that document P(R|x,q). By using the Bayes rule we get:

$P(R|x,q) = \dfrac{P(x|R,q)P(R|q)}{P(x|q)}$

Now, Bayes rule is as follows: $P(A|B) = \dfrac{P(B|A)P(A)}{P(B)}$

If you set $A := R$ and $B := x,q$ you get: $P(A|B) = P(R|x,q) = \dfrac{P(x,q|R)P(R)}{P(x,q)}$

If you compare the terms, you'll notice why I am confused:

• $x=x,q$
• $R,q=q$
• $R|q=R$
• $x|q=x,q$

This result has nothing to do with the initial claim. I think that I miss the definition of the 'comma' in this context. I am not aware of multi-dimensional probabilities. As I understand, a probability $P$ is always defined over a $\sigma$ algebra of an event space $\Omega$

In order to understand what's the idea behind the formula above, here a few things that could help:

• What does the comma precisely mean in the formula above (in mathematical notation)?
• What is the underlying $\Omega$ ? If there is a probability $P$, then there must be an Omega $\Omega$ which serves as the space on which we define probabilities. It's not clear at all what this space is. If a document is a vector $x \in \{{0,1}\}^n$ and the query is a vector $q \in \{{0,1}\}^n$, then defining a space like $\Omega := \{{0,1}\}^n \times \{{0,1}\}^n$ could make sense. In this case it's not clear what $R$ is. Maybe the intent is to use $\Omega := \{{0,1}\}^n \times \{{0,1}\}^n \times \{{relevant, nonrelevant}\}$
• Do $R$, $x$ or $q$ have anything to do with random variables? If yes, then it would help to see their domain, e.g: $R : \Omega \mapsto {0,1} $
• Because the conditional probability is defined between 2 sets, $R$ and $x,q$, as well as $x$ and $R,q$ or $R|q$ should represent sets. If $R$ is a random variable, then maybe in the formula above the term $R$ represents the set $\{\omega \in \Omega\ | R(\omega)=relevant\}$. What is then the set for $x$ or for $q$ ?

Answer 1

It's better to talk about x, q and R as (random) events - set of outcomes of the random experiment. x and q will be one element sets but R is an event denoting x is relevant to q and thus it is a subset of cartesian product X x Q (pair of a document and a query).

Comma is then set conjunction and P(A,B) = P(A ^ B) which equals to P(A) * P(B) when A is independent of B.

The wiki statement 'by using Bayesian rule' is a bit of a shortcut since you need to apply it several times. To derive the above formula for P(R|x,q), I would start with conditional probability definition (which is the root of Bayesian rule):

P(A|B) = P(A ^ B) / P(B)

Then:

P(R|x,q) = P(R ^ x ^ q) / P(x,q) = P(x|R,q) * P(R,q) / [ P(x|q) * P(q) ] =

= P(x|R,q) * P(R|q) / P(q) / [ P(x|q) * P(q) ]

When you divide nominator and denominator by P(q), you obtain

P(R|x,q) = P(x|R,q) * P(R|q) / P(x|q)