# Why is batch size limited by RAM?

The parameters of the network are changed to minimize the loss on the mini-batch, but usually the loss on the mini-batch is just the (weighted) sum of losses on each datum individually. Loosely, I would represent this as $$dT = \frac{1}{\text{batch_size}} \sum_{i \in \text{batch}} dT_i$$

Where $$dT$$ is the update of the net parameters for the batch and $$dT_i$$ is only for one training example. Why can't $$dT$$ be calculated 'on-line' then, where the only RAM needed is on the partial sum for $$dT$$ and whichever $$dT_i$$ you are working on at that moment?