this is more a curiosity query than anything else. The git repo for dolly gives us an easy way to swap, the training dataset , to train custom models, as long as we follow the format. I however have been through tons of content online BUT am unable to find ANY github repo which shows me exactly how the "instruction" and "context" are parsed in order to generate the final response. Here's my best guess
- use a pre trained model (gpt2 / gptj etc ) and create the mirror architecture. Now copy the weights from the pre trained model, hence initializing your custom model
- now pick the question & input text ( aka instruction and context ), convert them using a tokenizer ( choosing any HF based api ) and MOST IMP concatenate the "instrn" and "context" using some sort of separator ( this is what i read on using distilbert to solve QnA datasets )
- for the target / answer, we again, tokenize this thing and ensure that the final layer of the custom model outputs the same dimensionality as the temporal dimension of the output. Meaning if the "answer" is curtailed to 128 words, then the penultimate dimension of the output will be 128.
- try some version of cross entropy loss to compare these 2 and then we have the usual loss minimization exercise
i even read the InstructGPT paper and their git repo is basically an empty shell. Any pointers to git repos which show how this works, will be deeply appreciated.