# Partial Derivative - Pattern Recognition Bishop section 2.3

Please refer to page $$93$$, Pattern Recognition book by Bishop. This question is related to partial derivative of a term, the proof refers to appendix C.19 which states: $$\frac{\partial({a^{T}x})}{\partial{x}} = \frac{\partial({x^{T}a})}{\partial{x}} = a$$ and then moves on to take partial of normal probability distribution w.r.t. $$\mu$$ to get (eqn 2.120): $$\frac{\partial(x_{n} - \mu)^{T}\Sigma^{-1}(x_{n} - \mu)}{\partial{\mu}} = (x_{n} - \mu)$$

I am unable to see how we get the term. I tried checking elsewhere, but the explanation "consider $$(x_{n}-\mu)$$ to be scalar and $$(x_{n}-\mu)^{T}$$ as vector" does not resonate, as per my understanding both are vectors (as $$x_{n}$$ is vector) - it's their dot product that results in scalar.

I tried solving it step-by-step: Expand the term, ignoring $$\Sigma^{-1}$$ (contains no $$\mu$$ term) which gives: $$\frac{\partial{(x_{n}^{T}x_{n} - x_{n}^{T}\mu - \mu^{T}x_{n} + \mu^{T}\mu)}}{\partial{\mu}}$$ As per my understanding $$\frac{\partial}{\partial{\mu}}$$ of: $$x_{n}^{T}x_{n} = 0$$; of $$-x_{n}^{T}\mu = -\mu^{T}x_{n} = -\mu$$ but, not sure about $$\mu^{T}\mu$$

Given the context, I request help on partials :

1. Is $$\frac{\partial}{\partial{\mu}}$$[ $$x_{n}^{T}x_{n} = 0$$, $$(-x_{n}^{T}\mu = -\mu^{T}x_{n}) = -\mu$$]
2. $$\frac{\partial}{\partial{\mu}}(\mu^{T}\mu)$$
3. I have some understanding of calculus, so guide to minimum that'll help me derive equation 2.120.

Magnus & Neudecker will take time (450+ pages) and so I may have to gallop over another partial on page 94 $$\frac{\partial{(x_{n}-\mu)^{T}\Sigma^{-1}(x_{n}-\mu)}}{{\partial{\Sigma}}}$$

1. $$\frac{\partial}{\partial \mu} (x_n^Tx_n)=0,$$$$\frac{\partial}{\partial \mu}( -\mu^Tx_n)=-x_n$$
2. $$\frac{\partial }{\partial \mu}(\mu^T\mu) = 2\mu$$
3. Note that if $$A$$ is symmetric, then $$\frac{\partial }{\partial y}(y^TAy)=2Ay.$$
\begin{align}\frac{\partial }{\partial \mu}\left(-\frac12 (x_n-\mu)^T\Sigma^{-1}(x_n-\mu) \right)&=-\frac12\frac{\partial }{\partial \mu}\left( (\mu-x_n)^T\Sigma^{-1}(\mu-x_n) \right)\\ &=-\frac12\left(2\Sigma^{-1}(\mu-x_n) \right)\\ &= \Sigma^{-1}(x_n-\mu)\end{align}
• Got it! subject to my (mis)understanding - a $y^{T}y$ can be simply considered as $y^{2}$. And it $\frac{\partial}{\partial{y}}$ looks lot derivative chain. Correct me. – Continue2Learn Jun 12 '19 at 2:00
• well, that is the special case in $1$ dimension (of which it better be consistent), be careful when you write $y^2$ as vector squared is not well defined. – Siong Thye Goh Jun 12 '19 at 2:02
• ..pardon additional comment, for some some reason $\frac{\partial(a^{T}y)}{\partial{y}}$ turns $a^{T}$ to $a$. Isn't $a$ constant and so come out un-tampered by derivative - Bishop C.19. Regards. – Continue2Learn Jun 12 '19 at 2:07
• $a^Ty=y^Ta$. Note that there are two conventions in writing down the gradient, the column convention and the row convention. I use the column convention, hence I would write $a$. – Siong Thye Goh Jun 12 '19 at 3:32