Hello PyTorch experts:
Sentences and documents both can be variable length.
Lets say, we have following 2 docs:
(Here, each tensor is a sentence and each number in the tensor is an index of embedding matrix)
doc1=pad_sequence(doc1,batch_first=True) doc2=pad_sequence(doc2,batch_first=True) batch=pad_sequence([doc1,doc2],batch_first=True)
This will throw an error because the length of the longest sentence in the first document is greater than the length of the longest sentence in the second sentence.
So, we need to pad each sentence to the longest sentence of the batch.
One solution would be, find the length of the longest sentence in each batch and pad each sentence to that fixed length. But the pad_sequence function of PyTorch does not support that.
Am I missing something? Is there any other PyTorch way to do this?