I'm currently working on an extractive summary model based on Facebook's BART model. Consistent absolute length output would be highly desirable. The problem is that input length may vary wildly. That is to say, creating the training data, the instructions look like this:
- Take the input text (a news article) and start (recursively) deleting examples, excess details, unnecessary background information, quotes, etc.
- Once your summary has less than 90 words, stop deleting stuff.
- Fix up the text format to match the style guide.
The large BART model available on Huggingface was fine-tuned on 200 samples. All 200 samples had 60-88 words as the output sequence length. However, the model predicted outputs with lengths varying from 50 to 105 words, with some outliers as high as 120 words.
Now I'm questioning whether just throwing more samples at the problem will actually fix it. Since the model is doing very well following the style guide, I don't want to give up on this approach. The outputs which are too long can be eliminated by cranking up the length penalty. But that would make the "too short" case even more prevalent.
Can fine-tuning achieve a tighter range of output lengths just by specifying more examples? Or is there perhaps a more "hacky" solution to penalize output lengths not in the range?