# Derivates with respect to a vector

Suppose I have an equation, $f = X^TY + \dots$ (a few more terms), where $X$ is a vector and $Y$ is a matrix of appropriate dimensions, I want to know how can we take the derivative of $f \text{ w.r.t. } X$? I understand differentiation w.r.t one variable, but how does differentiation of another vector/matrix w.r.t a vector work?

Differentiating a function w.r.t a variable gives us the rate at with the function changes when we change the variable by a small amount. What does differentiating w.r.t a vector signify?

$$f =X^T Y$$ looks like this $$$$f= \begin{pmatrix} \sum_i x_iy_{i1}\\ \sum_i x_iy_{i2}\\ \vdots\\ \sum_i x_iy_{in}\\ \end{pmatrix}$$$$ where $$x_i$$ is the $$i^{th}$$ element of $$X$$ and $$y_{ij}$$ is the $$(i,j)^{th}$$ element of $$Y$$. Now, to get the gradient w.r.t $$X$$, is equivalent to deriving each element of $$f$$ w.r.t each element of $$X$$. This will lead to a 2D matrix: $$$$\nabla_X f = \begin{pmatrix} \frac{\partial}{\partial x_1} f_1 & \ldots & \frac{\partial}{\partial x_n} f_1\\ \vdots & \vdots & \vdots \\ \frac{\partial}{\partial x_1} f_n & \ldots & \frac{\partial}{\partial x_n} f_n \end{pmatrix}$$$$ We can therefore easily see that $$$$\frac{\partial}{\partial x_j} f_i = y_{ij}$$$$ Therefore $$\nabla_X f = Y$$.

Derivative of a univariate vector is the same as sum of derivatives of its component(Addition rule for differentiation).

As for an extra $Y$ matrix in multiplication with $X^T$, the derivative of $f$ is calculated using partial derivative rule.