I want to ask a fairly simple question I think. I have a deep background in pure mathematics, so I don't have too much trouble understanding the mathematics of the cost function, but I would just like to clarify what exactly the cost function is in a neural network in practice (i.e. implementing it on real datasets).
Given a fixed training sample, we can view the cost function as a function of the weights and biases, and thus optimizing this function is just finding the minimum of this function.
In practice, what is the cost function when you have thousands of training samples? Is it the sum of the cost functions over all the training examples?