What does this expression from gradient descent mean?

Question

I am looking over some neural network theory and came across this equation, coupled with this description (gradient descent ball-valley analogy):

''let's think about what happens when we move the ball a small amount Δv1 in the v1 direction, and a small amount Δv2 in the v2 direction. Calculus tells us that C changes as follows:''

I don't understand where this expression has come from. It looks like the product rule? Could someone either explain what it means and/or state the area of calculus this belongs to, so I can read up on it? Many thanks!

Answer 1

After reading up on multivariate calculus, I have understood what the above equation means. It is a simplified version of the multivariate chain rule - a rule used to differentiate multivariate functions. Here is what it looks like: $$\cfrac{dC}{dt}≈\cfrac{∂C}{∂v_1} \cfrac{dv_1}{dt}+\cfrac{∂C}{∂v_2} \cfrac{dv_2}{dt}$$

Time applies here because we are studying how the cost changes over time/over the course of training. Multiplying each side by $dt$ gives: $$dC≈\cfrac{∂C}{∂v_1} dv_1+\cfrac{∂C}{∂v_2} dv_2$$

And since the differential operator $d$ means a 'small change', we can replace this with delta to represent the same thing, thus arriving at the equation from the question: $$\Delta C≈\cfrac{∂C}{∂v_1} \Delta v_1+\cfrac{∂C}{∂v_2} \Delta v_2$$ To summarise, this equation is a simplified version of the multivariate chain rule, which although means the same as the original, does a better job in communicating that a 'small change' in $v_1$ and $v_2$ results in a 'small change' in $C$.