# BERT minimal batch size

Is there a minimum batch size for training/re-fining a BERT model on custom data?

Could you name any cases where a mini batch size between 1-8 would make sense?

Would a batch size of 1 make sense at all?

• In PyTorch, it is done easily by calling loss.backward() after each batch (the gradients get added to the existing ones) and calling optimizer.step() with optimizer.zero_grad() only in every k-th step. – Jindřich Nov 27 '20 at 10:42