Ignore or predict padding

Question

I have a sequence to sequence classification model with two classes (similar to NER transformer) and because my data samples have different lengths I use padding. Is it better to use a custom loss function (with a masking layer/attention mask) that ignores padding, or use an extra class for padding and classify padding as well? Or it doesn't make any difference in terms of model quality?

Answer 1

You should ignore padding. Predicting padding only adds a useless burden to the model.