0
$\begingroup$

I want to fine-tune an SLM like Phi-2 through the huggingface API. I am in doubt how to achieve that, because I see two ways to do that and I am wondering which is the correct way.

The task is just a sequence to sequence mapping. For the sake of simplicity let's just assume we do summarization.

There seem to be two possibilities to achieve this, and here I would like some clarification, if that is true and/or if there is a preferred way to do this.

Possibility 1:

Now, to get a head start I was looking around and trying to find scripts and tutorials. I found this one here, but I am genuinely confused, because it seems to me the author creates one single prompt with input and target sequence formatted in one place. There is no such thing as a label sequence that is expected to predicted but it's much more trained in a self-supervised or auto-encoder fashion. In code he writes the collate function that creates the dataset like this:

def collate_and_tokenize(examples):

    question = examples["question"][0].replace('"', r'\"')
    answer = examples["answer"][0].replace('"', r'\"')
    #unpacking the list of references and creating one string for reference
    references = '\n'.join([f"[{index + 1}] {string}" for index, string in enumerate(examples["references"][0])])

    #Merging into one prompt for tokenization and training
    prompt = f"""###System:
Read the references provided and answer the corresponding question.
###References:
{references}
###Question:
{question}
###Answer:
{answer}"""

    encoded = tokenizer(
        prompt,
        return_tensors="np",
        padding="max_length",
        truncation=True,
        max_length=2048,
    )

    encoded["labels"] = encoded["input_ids"]
    return encoded

During inference, logically, he then proceeds and feeds something like this:

new_prompt = """###System: 
Read the references provided and answer the corresponding question.
###References:
[1] For most people, the act of reading is a reward in itself. However, studies show that reading books also has benefits that range from a longer life to career success. If you’re looking for reasons to pick up a book, read on for seven science-backed reasons why reading is good for your health, relationships and happiness.
[2] As per a study, one of the prime benefits of reading books is slowing down mental disorders such as Alzheimer’s and Dementia  It happens since reading stimulates the brain and keeps it active, which allows it to retain its power and capacity.
[3] Another one of the benefits of reading books is that they can improve our ability to empathize with others. And empathy has many benefits – it can reduce stress, improve our relationships, and inform our moral compasses.
[4] Here are 10 benefits of reading that illustrate the importance of reading books. When you read every day you:
[5] Why is reading good for you? Reading is good for you because it improves your focus, memory, empathy, and communication skills. It can reduce stress, improve your mental health, and help you live longer. Reading also allows you to learn new things to help you succeed in your work and relationships.
###Question:
Why is reading books widely considered to be beneficial?
###Answer:
"""

where the model is asked to just generate answer. I get that, it makes sense, but I was expecting the following.

Possibility 2:

In the above mentioned tutorial the author just sets the labels equal to the input_ids. I would have done differently, such that I encode the target sequence and have that be my labels. This is how I feel like OpenAI's API works, but who knows what they are doing in the background ...


My questions all go in the direction of: what is correct? Is there a correct way? What is prefered? Do both approaches achieve the same thing?

Thank you. Please ask clarifying questions if I wasn't clear somewhere.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.