Why is there a $2$ at the denominator of the mean squared error function?

Question

In the famous Deep Learning Book, in chapter 1, equation 6, the Quadratic Cost (or Mean Squared Error) in a neural network is defined as

$ C(w, b) = \frac{1}{2n}\sum_{x}||y(x)-a||^2 $

where $w$ is the set of all weights and $b$ the set of all biases, $n$ is the number of training inputs, x is the set of all training inputs, y(x) is the expected output of the network for input x, and $a$ is the actual output of the network for input $x$, with respect to $w$ and $b$.

Most of this formula seems very clear to be, except the $2$ in the denominator. If I understand it correctly, we are summing up the squared vector length of (the actual output minus its expected output), for each training input (giving us the total squared error for the training set) and then divide this by the number of training samples, to get the mean squared error of all training samples. Why do we divide this by $2$ as well then?

In other places I've seen that Andrew Ng's lecture defines the Mean Square cost in a similar way, also with the $2$ in the denominator, so this seems to be a common definition.

Answer 1

This is just for mathematical convenience. When you differentiate $C(w,b)$, you will get an extra $2$. To eliminate that, $2$ is kept beforehand in denominator.

You can also watch this video on SVM lecture by Patrick Winston, where he uses a similar formula and then tells that he is using $2$ in denominator just for mathematical convenience.

Answer 2

1/2 is added just to make derivative simpler,when you take the derivative of the cost function that 2 in the power get cancelled with the 1/2 multiplier.

It doesn't matter if you even leave off that 2 and don't divide by 1/2 then you are just amplifying a scalar(single number) by 2, whereas what all you require is the underlying quantity(which is rate of change or slope), one thing to note in ML is learning linear quantity is simple and takes less time whereas amplifying by 2 can further lead to an extra step which is like asking the model to adapt to something extra.

Answer 3

It's just that it helps when we do the backprop of error while differentiating, as the 2 in the denominator cancels out with the 2 which we will get from the exponent..

So it's just a matter of Mathematical Convenience...