2
$\begingroup$

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch size has to be decreased to 1 image per batch, otherwise CUDA out of memory issues start to occur. While I know that this can create skewness in the training and it will likely be very hard to attain good convergence, can I ask this question in general to the experts: how valid is a batch size of 1 generally considered in image-based processing? The images in consideration can be considered large (high resolution). The optimizer used is Adam alongwith a warm up polynomial learning rate (with base around 0.00005), and 90k iterations.

(I understand that it would possibly be a good idea to try out a smaller crop size and bigger batch size, but would like to know the feedback from the community anyway)

$\endgroup$

1 Answer 1

0
$\begingroup$

After more research, I found that a batch size of 1 is quite common in deep-learning image processing use-cases where there are high memory/GPU requirements for model training. In fact, it gives better results at times.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.