# Conditional Multivariate Gaussian Distribution - Section 2.3, equation 2.74

Please refer section 2.3 (page 86-87) in Pattern Recognition and Machine Learning - Bishop

$$\mathit{N}(\mathbf{x}|\mathbf{\mu}, \Sigma)$$ where

$$\mathbf{x} = \begin{bmatrix} x_{a} \\ x_{b} \end{bmatrix}$$, $$\mathbf{\mu} = \begin{bmatrix} \mu_{a} \\ \mu_{b} \end{bmatrix}$$, $$\mathbf{\Sigma} = \begin{bmatrix} \Sigma_{aa} && \Sigma_{ab}\\ \Sigma_{ba} && \Sigma_{bb} \end{bmatrix}$$

The equation below expresses the Quadratic term of exponent in the Bivariate Gaussian probability distribution (ref. book eq. 2.70; ignoring $$-\frac{1}{2}$$):

$$(x - \mu)^{T}\Sigma^{-1}(x - \mu) = (x_a - \mu_a)^{T}\Lambda_{aa}(x_a - \mu_a) +(x_a - \mu_a)^{T}\Lambda_{ab}(x_b - \mu_b) +(x_b - \mu_b)^{T}\Lambda_{ba}(x_a - \mu_a) +(x_b - \mu_b)^{T}\Lambda_{aa}(x_b - \mu_b)$$

Author states "..conditional distribution $$p(x_{a} | x_{b})$$ can be evaluated from the the joint distribution $$p(x) = p(x_{a}, x_{b})$$ by fixing $$x_b$$ to the observed value and normalizing the resulting expression to obtain the valid probability distribution over $$x_{a}$$ .." and then filters out only the linear terms with $$x_{a}$$ and $$x_{a}^{T}$$ component. The result is presented in equation 2.74 as:

$$x_{a}^{T}\{\Lambda_{aa}\mu_{a} - \Lambda_{ab}(x_b - \mu_b)\}$$

I follow the steps, and see that above is due to $$x_{a}^{T}$$, $$x_{a}$$ in the first 2 terms of equation 2.70 - where "term(s)" refer to the components in 2.70 added using $$+$$ operator.

However, I seem to be getting two additions elements in $$x_a$$, in excess of the those present in 2.74 - due to the linear $$x_a$$ factor in $$3^{rd}$$ term (of equation 2.70). Excess term(s): $$(x_{b} - \mu_{b})^{T}\Lambda_{ba}x_a$$

Where am I going wrong? Please guide.

Note: I am taking the $$x_a$$ too to be linear component, whereas 2.74 is (seemingly) considering only the $$x_{a}^{T}$$ as linear term. Seems like I am losing track somewhere in the middle (perhaps in equation 2.71 - related extract, below):

$$-\frac{1}{2}(x - \mu)^{T}\Sigma^{-1}(x - \mu) = -\frac{1}{2}{x^{T}\Sigma^{-1}x} + {x^{T}\Sigma^{-1}\mu} + constant \ldots eq. 2.71$$

...where 'const' denotes terms which are independent of $${x}$$, and we have made use of the symmetry of $$\Sigma$$.

\begin{align}&-\frac12(x - \mu)^{T}\Sigma^{-1}(x - \mu) \\&= -\frac12(x_a - \mu_a)^{T}\Lambda_{aa}(x_a - \mu_a) \tag{1} \\&-\frac12(x_a - \mu_a)^{T}\Lambda_{ab}(x_b - \mu_b) \tag{2} \\&-\frac12(x_b - \mu_b)^{T}\Lambda_{ba}(x_a - \mu_a) \tag{3} \\&-\frac12(x_b - \mu_b)^{T}\Lambda_{aa}(x_b - \mu_b) \tag{4}\end{align}

Now let's focus on $$(1)$$, the linear part in $$x_a$$ are

$$-\frac12 (-x_a^T\Lambda_{aa}\mu_a - \mu_a^T\Lambda_{aa}x_a)=x_a^T\Lambda_{aa}\mu_a \tag{5}$$

Now focus on $$(2)$$, the linear part in $$x_a$$ is

$$-\frac12x_a^T\Lambda_{ab}(x_b-\mu_b) \tag{6}$$

Now focus on $$(3)$$, the linear part in $$x_a$$ is

$$-\frac12(x_b-\mu_b)^T\Lambda_{ba}x_a=-\frac12x_a^T\Lambda_{ab}(x_b-\mu_b) \tag{7}$$

There is no linear term of $$x_a$$ in $$(4)$$,

Adding $$(5)$$ to $$(7)$$, we have

\begin{align}x_a^T\Lambda_{aa}\mu_a-\frac12x_a^T\Lambda_{ab}(x_b - \mu_b)-\frac12x_a^T\Lambda_{ab}(x_b - \mu_b)&=x_a^T\Lambda_{aa}\mu_a-x_a^T\Lambda_{ab}(x_b - \mu_b)\\ &=x_a^T(\Lambda_{aa}\mu_a -\Lambda_{ab}(x_b-\mu_b))\end{align}

• Almost there! but missing something elementary - in eqn(5) and eqn(7), we're considering $A^{T}$ = A, (and moving ahead with calculation). This is something I am unable to recall...need help there – Continue2Learn Jun 10 '19 at 16:03
• $\Sigma$ is a positive definite symmetric matrix right? Hence $\Lambda$ is symmetrical as well. – Siong Thye Goh Jun 10 '19 at 16:10
• and please, the reason for $x_{a}$ = $x_{a}^{T}$ and $\mu_{a}$ = $\mu_{a}^{T}$. Thanks – Continue2Learn Jun 10 '19 at 16:17
• $x_a = x_a^T$ is not true. What I used is suppose $x^TAy=y^TA^Tx$ since both sides are scalars. – Siong Thye Goh Jun 10 '19 at 16:26
• Thanks. Closing the answer with my understanding that $(x_{a}\Lambda_{aa}\mu_{a}^{T}) = {(x_{a}^{T}\Lambda_{aa}\mu_{a})}^{T}$ (eq. (5)). And, L.H.S. = R.H.S. "Only because" the operation equals scalar values. Note: above reason, in current context, does not mean $\(AB)\^{T} = A\^{T}B\^{T}$ as Transpose operation is conducted on only one term of equation 5, LHS. – Continue2Learn Jun 10 '19 at 16:47