Partial Derivative - Pattern Recognition Bishop section 2.3

Question

Please refer to page $93$, Pattern Recognition book by Bishop. This question is related to partial derivative of a term, the proof refers to appendix C.19 which states: $$\frac{\partial({a^{T}x})}{\partial{x}} = \frac{\partial({x^{T}a})}{\partial{x}} = a$$ and then moves on to take partial of normal probability distribution w.r.t. $\mu$ to get (eqn 2.120): $$\frac{\partial(x_{n} - \mu)^{T}\Sigma^{-1}(x_{n} - \mu)}{\partial{\mu}} = (x_{n} - \mu)$$

I am unable to see how we get the term. I tried checking elsewhere, but the explanation "consider $(x_{n}-\mu)$ to be scalar and $(x_{n}-\mu)^{T}$ as vector" does not resonate, as per my understanding both are vectors (as $x_{n}$ is vector) - it's their dot product that results in scalar.

I tried solving it step-by-step: Expand the term, ignoring $\Sigma^{-1}$ (contains no $\mu$ term) which gives: $$\frac{\partial{(x_{n}^{T}x_{n} - x_{n}^{T}\mu - \mu^{T}x_{n} + \mu^{T}\mu)}}{\partial{\mu}}$$ As per my understanding $\frac{\partial}{\partial{\mu}}$ of: $x_{n}^{T}x_{n} = 0$; of $-x_{n}^{T}\mu = -\mu^{T}x_{n} = -\mu$ but, not sure about $\mu^{T}\mu$

Given the context, I request help on partials :

1. Is $\frac{\partial}{\partial{\mu}}$[ $x_{n}^{T}x_{n} = 0$, $(-x_{n}^{T}\mu = -\mu^{T}x_{n}) = -\mu$]
2. $\frac{\partial}{\partial{\mu}}(\mu^{T}\mu)$
3. I have some understanding of calculus, so guide to minimum that'll help me derive equation 2.120.

Magnus & Neudecker will take time (450+ pages) and so I may have to gallop over another partial on page 94 $$ \frac{\partial{(x_{n}-\mu)^{T}\Sigma^{-1}(x_{n}-\mu)}}{{\partial{\Sigma}}}$$

Answer 1

1. $$\frac{\partial}{\partial \mu} (x_n^Tx_n)=0, $$$$\frac{\partial}{\partial \mu}( -\mu^Tx_n)=-x_n$$

2. $$\frac{\partial }{\partial \mu}(\mu^T\mu) = 2\mu$$

3. Note that if $A$ is symmetric, then $$\frac{\partial }{\partial y}(y^TAy)=2Ay.$$

\begin{align}\frac{\partial }{\partial \mu}\left(-\frac12 (x_n-\mu)^T\Sigma^{-1}(x_n-\mu) \right)&=-\frac12\frac{\partial }{\partial \mu}\left( (\mu-x_n)^T\Sigma^{-1}(\mu-x_n) \right)\\ &=-\frac12\left(2\Sigma^{-1}(\mu-x_n) \right)\\ &= \Sigma^{-1}(x_n-\mu)\end{align}