Confused with the derivation of the gradient descent update rule

Question

I have been going over some theory for gradient descent. The source I am looking at said that the change in cost can be described by the following equation: $$∆C=∇C∙∆w$$ where $∇C$ is the gradient vector/vector derivative of the cost function (MSE) and $∆w$ is the change in weights. It said that the goal is to make the change in cost negative. Good so far. My issue is with the next part. It states that $$∆v=-η∇C$$ My issue is with this, and why $∆v$ is set to this. Why would we want to change the weights by a small amount of the gradient function?

Answer 1

Upon writing this I have realised the answer to the question. I am still going to post so that anyone else who wants to learn where the update rule comes from can do so. I have come to this by studying the equation carefully. $∇C$ is the gradient vector of the cost function. The definition of the gradient vector is a collection of partial derivatives that point in the direction of steepest ascent. Since we are performing gradient 'descent', we take the negative of this, as we hope to descend towards the minimum point. The issue for me was how this relates to the weights. It does so because we want to 'take'/'travel' along this vector towards the minimum, so we add this onto the weights. Finally, we use neta which is a small constant. It is small so that the inequality $∆C>0$ is obeyed, because we want to always decrease the cost, not increase it. However, too small, and the algorithm will take a long time to converge. This means the value for eta must be experimented with.