Conditional Multivariate Gaussian Distribution - Section 2.3, equation 2.74

Question

Please refer section 2.3 (page 86-87) in Pattern Recognition and Machine Learning - Bishop

$$\mathit{N}(\mathbf{x}|\mathbf{\mu}, \Sigma)$$ where

$ \mathbf{x} = \begin{bmatrix} x_{a} \\ x_{b} \end{bmatrix} $, $ \mathbf{\mu} = \begin{bmatrix} \mu_{a} \\ \mu_{b} \end{bmatrix} $, $ \mathbf{\Sigma} = \begin{bmatrix} \Sigma_{aa} && \Sigma_{ab}\\ \Sigma_{ba} && \Sigma_{bb} \end{bmatrix} $

The equation below expresses the Quadratic term of exponent in the Bivariate Gaussian probability distribution (ref. book eq. 2.70; ignoring $-\frac{1}{2}$):

$$(x - \mu)^{T}\Sigma^{-1}(x - \mu) = (x_a - \mu_a)^{T}\Lambda_{aa}(x_a - \mu_a) +(x_a - \mu_a)^{T}\Lambda_{ab}(x_b - \mu_b) +(x_b - \mu_b)^{T}\Lambda_{ba}(x_a - \mu_a) +(x_b - \mu_b)^{T}\Lambda_{aa}(x_b - \mu_b)$$

Author states "..conditional distribution $p(x_{a} | x_{b})$ can be evaluated from the the joint distribution $p(x) = p(x_{a}, x_{b})$ by fixing $x_b$ to the observed value and normalizing the resulting expression to obtain the valid probability distribution over $x_{a}$ .." and then filters out only the linear terms with $x_{a}$ and $x_{a}^{T}$ component. The result is presented in equation 2.74 as:

$$x_{a}^{T}\{\Lambda_{aa}\mu_{a} - \Lambda_{ab}(x_b - \mu_b)\}$$

I follow the steps, and see that above is due to $x_{a}^{T}$, $x_{a}$ in the first 2 terms of equation 2.70 - where "term(s)" refer to the components in 2.70 added using $+$ operator.

However, I seem to be getting two additions elements in $x_a$, in excess of the those present in 2.74 - due to the linear $x_a$ factor in $3^{rd}$ term (of equation 2.70). Excess term(s): $$ (x_{b} - \mu_{b})^{T}\Lambda_{ba}x_a $$

Where am I going wrong? Please guide.

Note: I am taking the $x_a$ too to be linear component, whereas 2.74 is (seemingly) considering only the $x_{a}^{T}$ as linear term. Seems like I am losing track somewhere in the middle (perhaps in equation 2.71 - related extract, below):

$$ -\frac{1}{2}(x - \mu)^{T}\Sigma^{-1}(x - \mu) = -\frac{1}{2}{x^{T}\Sigma^{-1}x} + {x^{T}\Sigma^{-1}\mu} + constant \ldots eq. 2.71$$

...where 'const' denotes terms which are independent of ${x}$, and we have made use of the symmetry of $\Sigma$.

Answer 1

\begin{align}&-\frac12(x - \mu)^{T}\Sigma^{-1}(x - \mu) \\&= -\frac12(x_a - \mu_a)^{T}\Lambda_{aa}(x_a - \mu_a) \tag{1} \\&-\frac12(x_a - \mu_a)^{T}\Lambda_{ab}(x_b - \mu_b) \tag{2} \\&-\frac12(x_b - \mu_b)^{T}\Lambda_{ba}(x_a - \mu_a) \tag{3} \\&-\frac12(x_b - \mu_b)^{T}\Lambda_{aa}(x_b - \mu_b) \tag{4}\end{align}

Now let's focus on $(1)$, the linear part in $x_a$ are

$$-\frac12 (-x_a^T\Lambda_{aa}\mu_a - \mu_a^T\Lambda_{aa}x_a)=x_a^T\Lambda_{aa}\mu_a \tag{5}$$

Now focus on $(2)$, the linear part in $x_a$ is

$$-\frac12x_a^T\Lambda_{ab}(x_b-\mu_b) \tag{6}$$

Now focus on $(3)$, the linear part in $x_a$ is

$$-\frac12(x_b-\mu_b)^T\Lambda_{ba}x_a=-\frac12x_a^T\Lambda_{ab}(x_b-\mu_b) \tag{7}$$

There is no linear term of $x_a$ in $(4)$,

Adding $(5)$ to $(7)$, we have

\begin{align}x_a^T\Lambda_{aa}\mu_a-\frac12x_a^T\Lambda_{ab}(x_b - \mu_b)-\frac12x_a^T\Lambda_{ab}(x_b - \mu_b)&=x_a^T\Lambda_{aa}\mu_a-x_a^T\Lambda_{ab}(x_b - \mu_b)\\ &=x_a^T(\Lambda_{aa}\mu_a -\Lambda_{ab}(x_b-\mu_b))\end{align}