I understand that all inputs in a batch need to be of the same size. However, it seems BERT/Transformers models can accept batches with different sizes as input.

How is that possible? I thought we needed to pad all examples in a batch to model.max_input_size, however, it seems HuggingFace does Dynamic Padding that allows sending batches of different lengths (till the time they are smaller than max_input_size)

enter image description here

Link: https://mccormickml.com/2020/07/29/smart-batching-tutorial/
Link2: https://huggingface.co/learn/nlp-course/en/chapter3/2?fw=pt#dynamic-padding



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.