Basically I am trying to create a context-needing question and long answer model and I was wondering what model would be best for such tasks, currently I am leaning towards T5, or GPT-NeoX-20B. Additionally for such tasks what datasets would be the most suitable, as of right now I have looked at CoQA, and SQuAD but none provide long answers.
Models that are good for long answer generation given context and question and what datasets would be the best for training?
$\begingroup$ How many words or characters has a long answer? GPT has a function min_lenght and max_lenght when you apply model.generate(...). So basically it can have any length. $\endgroup$– Nicolas MartinDec 28, 2022 at 15:54
$\begingroup$ I see, that works for me. $\endgroup$– Thamognya KodiDec 29, 2022 at 1:37
After the comment from Nicolas Martin, I found gpt2 for qa pair generation which gave reasonable steps on how to utilize Question and Answering for GPT models and then I can specify min_length and max_length to create a long answer.