2
$\begingroup$

I am trying to build a Q&A bot for which I have a bunch of documents like articles (specific domain).
I understand I can create a Retrieval-Augmented Generation (RAG) system for this, but I want to know how does fine-tuning work for this case, what would be the approach here?

Would it be creating a question-answer pairs (without context) manually or automated using llms example(gpt-4) and use a pre-trained model such as LLAMA-2 to fine-tune on this QA dataset? (Creating question-answer pairs would it mean I have to create thousands of question-answer pairs that would capture almost everything about the documents I have?)

Also, if I were to pre-train the model (LLAMA-2) on the documents I have and then fine-tune on the Question-Answer (no context) , would it yield better results?

Thank you for you time in advance.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.