I went through a few tutorials, examples recently, and all (not sure if just for demonstration purposes) done gradient descent for one input. To get a deep understanding of backpropagation, I wrote a program to do backpropagation just do understand it more deeply. In linear/logistic regression, it makes sense to do gradient descent on the average of the costs through multiple inputs, and outputs, because the there's just one layer of weights, and the inputs directly affect the outputs.
In case of neural networks, we get back a layer of activations (outputs), and we have the expected outputs with matching shape, so we're getting the costs, by subtracting the expected output, with our actual output, and we're propagating this back with the chain rule. But this way we have to compare our costs with the activations of the neurons, which are unique, and dependent on the inputs. So even if we would take a bunch of inputs, get their costs, and average the costs, how could we decide which neuron activation layer should it be compared to?