2
$\begingroup$

Objective

My goal is to fine-tune a pre-trained LLM on a dataset about Manchester United's (MU's) 2021/22 season (they had a poor season). I want to be able to prompt the fine-tuned model with questions such as "How can MU improve?", or "What are MU's biggest weaknesses?". The ideal responses would be insightful/logical and +100 words

Data

  • I will simply use text from the relevant wiki page as my data: https://en.wikipedia.org/wiki/2021%E2%80%9322_Manchester_United_F.C._season
  • How should I structure my data? Should it be a list dictionaries where the keys are the questions and the values are the answers (i.e. a list of question-answer pairs), or a long string containing all the text data (for context), or a combination of both?

Notes

  • I have mainly been experimenting with variations of Google's T5 (e.g.: https://huggingface.co/t5-base) which I have imported from the Hugging Face Transformers library
  • So far I have only fine-tuned the model on a list of 30 dictionaries (question-answer pairs), e.g.: {"question": "How could Manchester United improve their consistency in the Premier League next season?", "answer": " To improve consistency, Manchester United could focus on strengthening their squad depth to cope with injuries and fatigue throughout the season. Tactical adjustments could also be explored to deal with teams of different strengths and styles."}
  • Use of this small dataset (list of 30 dictionaries) has given poor results

Further Questions and Notes

  • Other than increasing the size of my dataset, is my approach sound?
  • What would you recommend as a minimum number of dictionaries to train/fine-tune the model on?
  • I am also aware that I can tune the hyperparameters to improve performance, but for now I am more concerned about my general approach being logical
$\endgroup$
1
  • $\begingroup$ Please, update your question if you make any progress in your project. $\endgroup$
    – Cloud Cho
    Commented Oct 4, 2023 at 19:34

1 Answer 1

1
$\begingroup$

Check the steps at the Huggingface beginner's guide Question Answering with SQuAD 2.0 to begin with a normal question answering model. Have some look at the Fine-tuning guide at OpenAI as well (which tells you not to fine-tune at all if you have not tried prompt engineering).

If you have more than one question, you have to train with a json with the right tree (if you have answers, you can put them there like the questions, with "id": 1):

{
    "context": "xyzxyzxyzxyzxyzxyzxyzxyzxyzxyzxyz",
    "questions": [
        {"question": "xyz", "id": 1},
        {"question": "xyzxyzxyzxyz", "id": 2},
        {"question": "xyzxyz", "id": 3},
        {"question": "xyzxyzxyz", "id": 4}
    ]
}

Now that is the pattern for just one document, and you can put other nodes in there like sources, meta_data, it is up to you how you name them and how you preprocess that wiki site, and put the nodes in this nested tree like above. If you take just that one wiki page, you have just one document that you want to ask questions to, but you might want to ask questions on chosen chapters only or to other websites, then you need to put a parent node to the given example and the model will understand this.

You can ask open questions, then you just do not put any answers in the json. Or you know the answers and want to train the chat bot that the known answers are the output, then train the Question Answering model with question-answer pairs instead.

Thus, you should put the full context of Wikipedia there (one time, not for each question), that will make the output much better. You can then put many questions against it, decide for each question whether it should learn an answer, and train the model as in the guide.

Mind that asking questions without training the answers will only cut text from the context as an answer. There is no free-speech answer from that. If you want free-speech answers, you should check Which techstack and IDE are good to set up a RAG model? (=Retrieval-augmented generation models jointly fine-tune DPR and sequence-to-sequence models).

I have not done the RAG model myself, and I also do not know when the Question Answering model will begin to generalize on unknown questions if you make a large input of question-answer pairs instead. Here are the Example count recommendations of the OpenAI site

To fine-tune a model, you are required to provide at least 10 examples. We typically see clear improvements from fine-tuning on 50 to 100 training examples with gpt-3.5-turbo but the right number varies greatly based on the exact use case.
We recommend starting with 50 well-crafted demonstrations and seeing if the model shows signs of improvement after fine-tuning. In some cases that may be sufficient, but even if the model is not yet production quality, clear improvements are a good sign that providing more data will continue to improve the model. No improvement suggests that you may need to rethink how to set up the task for the model or restructure the data before scaling beyond a limited example set.

Most of what I write is from the Huggingface guides and the questions on Stack Exchange and some chats with ChatGPT 3.5.

Academic paper on the needed counts for fine-tuning question-answering models

See GenAI SE How do I "teach" a large language model new knowledge? which links to the LIMA: Less Is More for Alignment paper. If I read that summary right, this paper shows that it is easy to fine-tune the big model further with your own small dataset. Since in that 5/2023 paper, they

fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses.
... LIMA demonstrates remarkably strong performance, learning to follow specific response formats from only a handful of examples in the training data, including complex queries that range from planning trip itineraries to speculating about alternate history. Moreover, the model tends to generalize well to unseen tasks that did not appear in the training data.

In other words, and I am still hoping that I read it right, fine-tuning with a Question Answering model can work very well with a sample of 1000 questions and answers. With such a count, it can even generalize from the input answers and answer other untrained questions.

That would mean that 50 to 100 pairs can generalize, but 1000 pairs will build a very good generalizing model. That is a lot, but on the other hand, it can be done.

Natural generalizability

This altogether sounds like the generalizability of empirical statistics as such. There, you also make the biggest step if you ask 100 people or have whatever count of 100 in sociology or nature. The rest will only make a small step further.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.