I'm wondering why we multiple by 2 when calculating partial derivatives.

I'm referencing the 2's that I've circled below, from here.

enter image description here

We also see this in the python implementation, m_deriv += -2*X[i] * (Y[i] - (m*X[i] + b)), there is a 2.

How can I understand the intuition here?

My math is not strong so apologies in advance if this is a silly question.


You're using the Mean Square Error $\Sigma\frac{1}{N}(y-(Wx+b))^2$as the loss function, if you take the derivative, you will have the $2$. In some materials, we will use $\frac{1}{2}\Sigma\frac{1}{N}(y-(Wx+b))^2$ as the loss function to cancel out the $2$. In fact, this doesn't matter at all and it has no impacts on params optimization.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.