26
$\begingroup$

I'm not sure why you need to multiply by $\frac1{2m}$ in the beginning. I understand that you would have to divide the whole sum by $\frac1{m}$, but why do we have to multiply $m$ by two?

Is it because we have two $\theta$ here in the example?

$\endgroup$
1
  • $\begingroup$ I think, just for the conviniency of further derivation of squared error cost: 1/(2m)*loss**2 derivative becomes 1/m*loss because 2/2m=1/m - no further division of quantity of sample by halfs or parts $\endgroup$
    – JeeyCi
    Feb 22 at 6:55

2 Answers 2

24
$\begingroup$

It is simple. It is because when you take the derivative of the cost function, that is used in updating the parameters during gradient descent, that $2$ in the power get cancelled with the $\frac{1}{2}$ multiplier, thus the derivation is cleaner. These techniques are or somewhat similar are widely used in math in order "To make the derivations mathematically more convenient". You can simply remove the multiplier, see here for example, and expect the same result.

$\endgroup$
2
  • $\begingroup$ Thanks for your clean answer! But doesn't this affect the result at all? $\endgroup$ May 18, 2019 at 14:11
  • 3
    $\begingroup$ Not at all. It is a simple constant. After it is a convex function, and will lead to a global minima! $\endgroup$ May 18, 2019 at 15:13
7
$\begingroup$

It makes the math easier to handle. Adding a half or not doesn't actually matter since minimizing is unaffected by constants.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.