1
$\begingroup$

$$ C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $$

Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it be $\frac{1}{n}$ instead?

Link

$\endgroup$

2 Answers 2

3
$\begingroup$

This is really just like a convention that appears in some places because we normallt want to take the derivative of the cost function (i.e. compute gradients), which means the power of 2 would be taken to the front.

If we put the $\frac{1}{2}$ at the front to begin with, it just looks nicer once we have finished. I have seen this written somewhere before in a paper, but can't find a reference right now.

Because the nominal values of cost themselves (the scale of the values) is not of importance, we can scale it as we like really. Multiplying by a constance of $0.5$ does not change the algebraic behaviour.

$\endgroup$
2
  • $\begingroup$ Thanks. That does make sense. Could you please leave the link if you happen to remember it? $\endgroup$ Jun 13, 2018 at 15:12
  • 1
    $\begingroup$ @AniketChowdhury - I will add any links I can find - or maybe someone else can add links as a comment :) $\endgroup$
    – n1k31t4
    Jun 13, 2018 at 16:41
1
$\begingroup$

It's pretty common to formulate quadratic loss in that way, because $\frac{d}{dx}\frac{(x-x')^2}{2} = x - x'$.

If you're only interested in minimizing that quantity then it doesn't matter if you rescale it by a constant, since locations of extrema would be the same.

$\endgroup$
1
  • $\begingroup$ Yeah. Thanks. I should have figured that out. Sorry for the bother. $\endgroup$ Jun 13, 2018 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.