I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch size has to be decreased to 1 image per batch, otherwise CUDA out of memory issues start to occur. While I know that this can create skewness in the training and it will likely be very hard to attain good convergence, can I ask this question in general to the experts: how valid is a batch size of 1 generally considered in image-based processing? The images in consideration can be considered large (high resolution). The optimizer used is Adam alongwith a warm up polynomial learning rate (with base around 0.00005), and 90k iterations.
(I understand that it would possibly be a good idea to try out a smaller crop size and bigger batch size, but would like to know the feedback from the community anyway)