# Why do we double the number in a quadratic cost function or MSE?

$$C(w,b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2$$

Where y is a 10-dimensional vector, a is the output, w is the weight and b is the bias and n is the number of inputs. If this is the MSE, shouldn't it be $\frac{1}{n}$ instead?

This is really just like a convention that appears in some places because we normallt want to take the derivative of the cost function (i.e. compute gradients), which means the power of 2 would be taken to the front.

If we put the $\frac{1}{2}$ at the front to begin with, it just looks nicer once we have finished. I have seen this written somewhere before in a paper, but can't find a reference right now.

Because the nominal values of cost themselves (the scale of the values) is not of importance, we can scale it as we like really. Multiplying by a constance of $0.5$ does not change the algebraic behaviour.

• Thanks. That does make sense. Could you please leave the link if you happen to remember it? Jun 13, 2018 at 15:12
• @AniketChowdhury - I will add any links I can find - or maybe someone else can add links as a comment :) Jun 13, 2018 at 16:41

It's pretty common to formulate quadratic loss in that way, because $\frac{d}{dx}\frac{(x-x')^2}{2} = x - x'$.

If you're only interested in minimizing that quantity then it doesn't matter if you rescale it by a constant, since locations of extrema would be the same.

• Yeah. Thanks. I should have figured that out. Sorry for the bother. Jun 13, 2018 at 15:13