3
$\begingroup$

Suppose I have an equation, $f = X^TY + \dots$ (a few more terms), where $X$ is a vector and $Y$ is a matrix of appropriate dimensions, I want to know how can we take the derivative of $f \text{ w.r.t. } X$? I understand differentiation w.r.t one variable, but how does differentiation of another vector/matrix w.r.t a vector work?

Differentiating a function w.r.t a variable gives us the rate at with the function changes when we change the variable by a small amount. What does differentiating w.r.t a vector signify?

$\endgroup$
1
$\begingroup$

$f =X^T Y$ looks like this \begin{equation} f= \begin{pmatrix} \sum_i x_iy_{i1}\\ \sum_i x_iy_{i2}\\ \vdots\\ \sum_i x_iy_{in}\\ \end{pmatrix} \end{equation} where $x_i$ is the $i^{th}$ element of $X$ and $y_{ij}$ is the $(i,j)^{th}$ element of $Y$. Now, to get the gradient w.r.t $X$, is equivalent to deriving each element of $f$ w.r.t each element of $X$. This will lead to a 2D matrix: \begin{equation} \nabla_X f = \begin{pmatrix} \frac{\partial}{\partial x_1} f_1 & \ldots & \frac{\partial}{\partial x_n} f_1\\ \vdots & \vdots & \vdots \\ \frac{\partial}{\partial x_1} f_n & \ldots & \frac{\partial}{\partial x_n} f_n \end{pmatrix} \end{equation} We can therefore easily see that \begin{equation} \frac{\partial}{\partial x_j} f_i = y_{ij} \end{equation} Therefore $\nabla_X f = Y$.

$\endgroup$
-1
$\begingroup$

Derivative of a univariate vector is the same as sum of derivatives of its component(Addition rule for differentiation).

As for an extra $Y$ matrix in multiplication with $X^T$, the derivative of $f$ is calculated using partial derivative rule.

Please see the link here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.