Why would we add regularization loss to the gradient itself in an SVM?

Question

I'm doing CS 231n on my own. I'm looking at this solution to a question that implements a SVM.

Relevant code:

# average the loss
loss /= num_train
# average the gradients
dW /= num_train
# add L2 regularization to the loss
loss += reg * np.sum(W * W)
# ????
dW += 2 * reg * W

I don't understand why we would add regularization loss to the gradient. My understanding of regularization is we use it to prefer certain weights, $W$, over others. But... I don't understand

1. What type of regularization is occurring to dW (L2 regularization operates on the square of all values of the weights -- this is not squaring anything)
2. Why we would tweak the weights themselves, presumably you want to tweak the loss which will incentivize changing the weights in a certain direction. Why would you tweak the weights (well, their gradients) themselves?

Answer 1

The l2 regularization term is being added to the loss itself. But then you need to find the gradient of this new loss; since gradients are additive, this is the same as the gradient of the unpenalized loss plus the gradient of the l2 term, the latter of which is the quantity specified in the last line of code.

Note that it makes sense: when updating the weights, you will subtract some multiple of the gradient, so are moving the weights opposite their current location, i.e. toward the origin, as you expect regularization to accomplish.